well if you hanv't been charged yet then just cancel your order. If you have been one of the very select and lucky few who have been charged then your Rift should be showing up either tomorrow or sometime during the week. And in which case you would have to contact support about returning the Rift and getting a refund.
The funds generally will sit in your paypal account until you allocate them somewhere else. Send the funds to a bank account or what not. So yes, you can. But only if you Rift has shipped, otherwise just cancel and you won't get charged.
But Oh, how bitter a thing it is to look into happiness through another man's eyes.