Skip to main content
ginestopo
Up and Coming
November 20, 2023

Passthrough camera data is not available on Quest 3 developing with Unity

Basically it is not possible to access the real time passthrough video from Quest 3 in Unity.

Many devs among the internet think it is a crucial feature to be able to process to create incredible mixed reality apps with object recognition using machine learning. Nevertheless, it is not possible to access the real time video captured by quest 3 in unity.

 

Here is a reddit post made by me where people are requesting this feature.

 

Please, we need access to real time passthrough video in Unity or other engines.

    109 replies

    Show previous replies
    Ivanaa
    Participant
    February 9, 2025

    Unfortunately there haven't been any updates on Meta's release of the camera API other than Andrew Bosworth (head of Meta's Reality Labs) confirming on Instagram (February 3rd, 2025) that it is still planned for early 2025.

    In other news though, Google has confirmed that their upcoming OS Android XR will allow developers with camera access, which will work similarly to what is currently done on Android where the app requests camera access and the user must allow access. Though there is no confirmed release date for Samsung's Project Moohan headset, which will run on Android XR, it is expected to release this year.

    Hopefully Meta releases their camera API soon or else they will be catching up to even more companies on this front, like Pico 4 Enterprise, Apple Vision Pro, the previous professional AR headsets (HoloLens 2, Magic Leap 2, Vuzix, etc.), all of which already allow camera access to mostly enterprise developers, and now Google/Samsung.

      Ivanaa
      Participant
      March 4, 2025

      Meta added headset camera permission description to its documentation 3 days ago - https://www.meta.com/help/quest/511837681585017/

      It mentions that users must allow camera access permission, which will allow the app to access the camera and what is in front of the headset (not including any virtual objects/content). The document provides some examples of possible camera uses (object tracking, location-based, and using machine learning to process camera feed). It also mentions access to real-time camera feed (which I am assuming and hoping means raw access) and intrinsic camera values (camera focal length, principal point, image dimensions and camera position relative to center of device).

      I did not find any mention of camera access in their document on viewing and managing permissions (which was updated 5 days ago), so maybe there are still more things to be updated first before release: https://www.meta.com/help/quest/337925588963509/


      At least it looks like the wait won't be too much longer. I'm hoping it comes out before the end of April or else it won't be early 2025 anymore.

      Update: Meta will be giving some updates on their camera access at GDC.

        Ivanaa
        Participant
        March 14, 2025

        We finally now have camera access on the Quest 3(S) \o/

        See documentation: https://developers.meta.com/horizon/documentation/unity/unity-pca-documentation

        Sample project/code from Meta: https://github.com/oculus-samples/Unity-PassthroughCameraApiSamples

        A video has already been created showing how to access it in Unity: https://youtu.be/1z3pcMJbnRA?feature=shared

        Tutorial on getting it working on a new/empty project: https://skarredghost-com.cdn.ampproject.org/c/s/skarredghost.com/2025/03/17/how-to-camera-access-meta-quest-3-unity-6/amp/

        List of projects using camera access created by the community: Here is a list of what some developers have already created with the Quest camera access - https://github.com/xrdevrob/QuestCameraKit?tab=readme-ov-file#community-contributions


        Current Limitations

        1. No Unity Editor Camera Access: Unfortunately there isn't a way to access the camera via the Unity Editor, which was a useful feature with other AR devices, like the HoloLens, to speed up development.
        2. Apps can't be published on the official store: Applications can't be posted on Meta's store for now. Not sure if you can create it for desktop/pcvr though, in case you need to more processing and rendering power.
        3. Camera Resolution not Full-HD: The camera resolution isn't great, only 1280 x 960.
        4. Low/Mismatched Data Rate: Only 30Hz, much less than the refresh rate of what is being rendered and calculated.
        5. No Raw Depth Sensor Access? I'm not sure if we have raw access to the depth sensor, for instance for point cloud generation.

        Hopefully these things will be improved in future software and hardware updates.

          dariomaciel
          Honored Contributor
          March 17, 2025

          just commenting here so nobody forget about Unreal devs please!!!
          passthrough API for unreal!!

            Terms & ConditionsAccessibility statement