Forum Discussion
Reverse AR (Augmented Reality): bringing the real into VR
I was trying to figure out a way for a user to see their keyboard while wearing a Rift. Would this simply be Augmented Reality in reverse? Instead of having digital elements being projected into the “...
Hands are reconstructed as 3D mesh from depth camera (and textured using color from color camera).
Currently I'm having custom filter that removes everything except hands. It takes into notice HMD orientation.
Filter is based on depth discontinuity, so it will properly display not only your hands but everything you will hold in them.
I've made few tests and I was able (barely but still) to read text messages from my smartphone in VR ;].
Keyboard should be visible as well if you will just look down and touch it, but there are technical limitations.
Prototype of depth sensors I'm using is measuring depth using IR. That IR beams re reflected in all directions when they hit glossy (?) plastic surface of my black keyboard. This results in disappearing keys as IR beam is not returning to the sensor and samples are removed as bad. With better depth camera HW this could look much better, without loosing samples.
Currently I'm having custom filter that removes everything except hands. It takes into notice HMD orientation.
Filter is based on depth discontinuity, so it will properly display not only your hands but everything you will hold in them.
I've made few tests and I was able (barely but still) to read text messages from my smartphone in VR ;].
Keyboard should be visible as well if you will just look down and touch it, but there are technical limitations.
Prototype of depth sensors I'm using is measuring depth using IR. That IR beams re reflected in all directions when they hit glossy (?) plastic surface of my black keyboard. This results in disappearing keys as IR beam is not returning to the sensor and samples are removed as bad. With better depth camera HW this could look much better, without loosing samples.