What would it take to build a VR editor?

Would a plug-in be enough to use the oculus rift as my scene view and use keyboard shortcuts for the rest of the screen?

I'm new to Unity and Maya (been using them everyday for 2 weeks) and have begun to get into the rhythm of manipulating polygons to make basic shapes and objects and then placing them in a level. I can't help but think that it would be cool to be able to manipulate the objects I'm looking at while I'm wearing the rift.