Forum Discussion
Accessing Video-Passthrough
Are there any plans on letting Devs access the Quest's Video-Passthrough capabilities via SDK? I'd love to mess around with AR / MR Applications. I know that this is possible on the Vive Pro utilizing...
Wow! They're finally going to give us access to the passthrough stream to do AR!
Oculus Quest v31 Adds Experimental Passthrough API (uploadvr.com)
Kind of.
There is no direct access to the stream. Instead you can choose to render passthrough layers, but have no access to the actual images. From the Oculus developer blog:
"We built Passthrough API with privacy in mind. Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device."
https://developer.oculus.com/blog/mixed-reality-with-passthrough/
So you could do something like show the real life floor around a player, but not do things like fiducial marker tracking or depth mapping.
Yeah - regarding the marker tracking, I'm thinking the camera resolution isn't high enough to read the markers anyway, unless they are pretty large. But, if you look at the video where he draws the box in white lines, his arm occludes the lines behind it - I'm guessing there will be some sort of depth capabilities, even if it's just handled internally by the API. In general, I wasn't expecting true, high rez AR capabilities... I know the hardware wasn't built for it (hopefully the next one will!). I'm just excited to finally get to play around with the basics - y'know... put a virtual dartboard on the wall, throw some virtual darts at it - and similar simple experiments. I have a ZED Mini camera that I mounted to the front of my Rift S, but there were so many issues trying to get it all to work together that I eventually stopped going down that road.
