Forum Discussion
Has this been done yet???
Has anyone made an AR type app for Rift that would allow for the use of a webcam, kinect cam, or any other cam in order to see your monitor through the Rift and place AR elements in and around the view? That seems like a pretty simple solution to the whole extended display issue. I read something about one of the droid vr's using the onboard cell cam for viewing the real world. Figure it must have been done already and maybe burried in the posts somewhere?
13 Replies
- I think you can do that with Leap Motion's Oculus Passthrough feature.
- @MrsVR For this type of application, it's important to note that you'd need a virtual monitor. The ability to see your monitor directly will be possible with our Dragonfly embedded sensor, but the current generation of Leap Motion technology provides exclusively infrared video -- so that monitors appear black through the video feed.
"alexcolgan" wrote:
@MrsVR For this type of application, it's important to note that you'd need a virtual monitor. The ability to see your monitor directly will be possible with our Dragonfly embedded sensor, but the current generation of Leap Motion technology provides exclusively infrared video -- so that monitors appear black through the video feed.
:o I've been hesitant to take the leap motion plunge but this would make me take the plunge :-) Any ETA on the next gen? I suppose even that could be skipped so long as we had the ability to see a keyboard. I normally prefer to use a mix of Razer Orbweaver (which has a green backlite), and Logitech k810 (which has a white backlite). Is there any frequency of light that would pass through at this time?
Thanks!- No ETA on Dragonfly yet, but I can confirm that the letters on my keyboard are visible in infrared (it's nothing special, just your average keyboard).
The trick is that hand tracking becomes unreliable when your hands get close to complex backgrounds (like keyboards ;) ), so an app of this kind would need to ignore hand tracking when the hands are near the keyboard. Perhaps an "inactive zone" (around the keyboard) and an "active zone" (areas of open space where you could place and manipulate things). It would be a really cool application to see brought to life :D "alexcolgan" wrote:
No ETA on Dragonfly yet, but I can confirm that the letters on my keyboard are visible in infrared (it's nothing special, just your average keyboard).
The trick is that hand tracking becomes unreliable when your hands get close to complex backgrounds (like keyboards ;) ), so an app of this kind would need to ignore hand tracking when the hands are near the keyboard. Perhaps an "inactive zone" (around the keyboard) and an "active zone" (areas of open space where you could place and manipulate things). It would be a really cool application to see brought to life :D
No doubt that would be awesome. Short of a bad smoking habit I would have no reason to ever take off the rift haha!"alexcolgan" wrote:
@MrsVR For this type of application, it's important to note that you'd need a virtual monitor. The ability to see your monitor directly will be possible with our Dragonfly embedded sensor, but the current generation of Leap Motion technology provides exclusively infrared video -- so that monitors appear black through the video feed.
That is some sexy news!! We can live in our Rifts then!
Since you're here: Leap Motion has a tendency to get confused and will re-position the virtual hands. Is this something we can expect to be improved in the next gen?"MrsVR" wrote:
Since you're here: Leap Motion has a tendency to get confused and will re-position the virtual hands. Is this something we can expect to be improved in the next gen?
You can expect major tracking improvements, though not exclusively due to hardware -- the really fundamental problems ahead for our engineers lie in further optimizations and refinements to the software. You can have the best cameras in the world, but it's always going to be the software that needs to convert pixels into hand data fast enough that human beings can't perceive motion-to-photon latency.
Fortunately, ongoing improvements to the core software will benefit everything from the current peripheral, to upcoming VR headsets, to automotive modules like Meadowhawk. Even now, today's hardware is a good baseline for experimenting with the kinds of APIs and resources that will really come into their own with Leap-embedded VR headsets. These will feature modules with 64-mm camera spans (matching the average human eyespan), massively improved FOV, and greater-than-HD cameras.
Our CTO actually gave a talk a little while back about what the future of Leap + VR may look like; the video and slides are here if you're interested: http://blog.leapmotion.com/david-holz-quick-peek-future-wearable-displays-inputs/- wow, dragonfly looks to be precisely what we need. Granted most communications are leading towards voice with Cortana and so on...I think we've all had many situations where the keyboard just gets a particular job done quickly. Simple stuff like Steam chat in game, or an important email comes in while in the middle of an important project and you don't have time to fiddle with voice corrections. As you are testing currently, do you see the latency dropping to the point were you could type as fast as normal without being distracted by the delayed vision in the HMD? We need to fast forward to your CTO's gen 3 HMD concepts so I can implement my electrical background and my smart home ideas :-)
- Indeed! Currently, with a properly optimized setup, the latency for video passthrough on our end is a little over 8 milliseconds. The overall latency of the system (e.g. including that of the VR headset itself) is going to continue to come down over time.
- Dragonfly sounds awesome!
I have another question: Do you know if Leap Motion and Sixense STEM is a good combo? Do they play nice together?
