Access Camera feed / Scan a QR Code with Quest 3
Hi,
I was wondering if, unlike the Quest Pro, the Quest 3 would allow access to the camera feed (RGB or IR, or both), and/or allow us to scan a QR code with the cameras ? That would be great.
Thanks
Forum Discussion
13 Replies
I believe it is currently not possible. Hopefully it will be eventually. Otherwise it's a huge missed opportunity for a XR/AR device like this! I often hear people say that the main issue with the quest 3 XR is the lack of software utilizing it. If Meta would allow access to camera data, a lot of computer vision and AR apps that are currently available for Android phones could potentially work on the headset. Apps involving face tracking to add funny effects/filters to people's faces, Google maps Live View for navigation, various applications for QR code reading/tracking like you mentioned and even computer vision based object detection and classification.
Sure, privacy is an important consideration, but this can be ensured by simply having apps ask the user for permission, similar to what they already do with microphone and scene data.
Yeah, hopefully they will reply to this saying it's on the way
I just published a post requesting this same feature. It think the lack of this feature is destroying the incredible potential of Quest 3 and mixed reality. I wanted to create an object recognition app using passthrough cameras and Unity.
Please have a look and support so they can implement this feature soon 🙂
Our application has a legitimate reason to need access camera feed.
Mixed reality experiences that implement computer vision algorithms to react to things in the real world.
Hopefully it will be eventually.
My team also needs it for computer vision applications that could make quest 3 product shine, which is a huge benefit for Meta
Hello, same here for me, I need a QR scanning capabilities to allow user to quickly map quest3 device with other user devices.
What I am thinking of now as a workaround would be to ask the user to make a screenshot (meta button + right controller trigger) and have the app discover the fresh saved screenshot image png in the Oculus/Screenshots folder. The screenshot filenames contain the app bundleId so there should be no problem filtering out the newest created image. Hope that helps some of other devs 🙂Hi,
This could work for some apps I guess. Our app doen't require the use of controllers so it's not really what we want, but that could be a workaround.
Exactly. We need camera color data access.
It could be a workaround. Nevertheless, it would probably cause a poor UX and doesn't solve the needs discussed in this forum. We need camera access to create more complex and actually useful applications. Having access to camera color data would make the process you are trying to do very smooth.
We have also been developing many vision applications on HoloLens 2, Quest 3 is an excellent candidate to replace. Camera access is very useful in this case, hundreds of applications as well as our customers are also waiting for this.
We are also working on several computer vision applications which would at least need access to some coarse information, QR and Aruco codes would already help a lot. Thus I actually had a call with Meta a couple of weeks ago but the outcome wasn't very constructive, just evading responses like "Don't know", 'Can't tell you", blabla, definitely no interest about B2B as it seemed.
Really sad actually though of course the camera quality of the Quest 3 is as of now still extremely terrible in my opinion but nontheless we would like to prepare already for Quest 4 or Quest 2 Pro.
Until then we will likely use some work around by mapping camera data from other cameras in the room into the world space of the AR headset, dirty, but works.You can now access the Meta Quest's Passthrough Camera: https://www.youtube.com/watch?v=1z3pcMJbnRA
See samples here: https://github.com/xrdevrob/QuestCameraKit
It contains a QR code sample as well! 🙂
Official Passthrough Camera Samples: https://github.com/oculus-samples/Unity-PassthroughCameraApiSamples
Documentation: https://developers.meta.com/horizon/documentation/unity/unity-pca-overviewCan I also use it with XR Interaction toolkit and AR Core or am I forced to use Meta SDK?
