Skip to main content
NEMO54174
New Participant
December 6, 2023

How can I get a depth map(or point cloud) just from my quest 3's depth sensor???

  • December 6, 2023
  • 36 replies
  • 18779 views

I know there is a depth sensor on the quest 3 and I want to use its depth data for further image research, but my MQDH only supports recording video from the binocular camera. 😞 I've checked the documentation for the Depth API, and it seems that the depth information has been encapsulated many times, and I wonder if there is a way to get the depth map or point cloud image directly, thank you very much!!🙏

    36 replies

    Show previous replies
    FunGamePlayer
    Participant
    January 13, 2025

    The texture size I get in the scripts is 2000x2000 but I've read something about 512x512 in another post that also used a compute shader. Since I get the texture in a script (from the global shader variables) and give it to the VFX Graph to use, I don't know if accessing the texture results in a different size inside compute shaders. But since there are other posts online that talk about using the texture inside a compute shader with the size being 512x512 maybe thats just something that happens automatically. Anyways I don't really know why this would be happening but does it matter? 

    Aethoen
    New Participant
    February 5, 2025

    Sorry to bother you again, but have you been able to use the DepthAPI after Meta updated to v72? After taking a break from this project for the past few weeks, I was unable to enter play mode on Unity and was stuck on a loading screen. This was after updating the drivers for Quest Link. Thanks for your help.

    FunGamePlayer
    Participant
    February 5, 2025

    I'm not quite sure what your problem is but it could be related to an issue I had after v72. Up until now I only tested the Depth API build directly to the quest 3 and not in the editor while connected via quest link and thus don't know if my issue could lead to the same result if tested in the unity editor.
    In v72 it seams that some things related to the creation of the depth data provided by the Depth API have changed either unknowingly or undisclosed by meta. In my case the data was either empty (only ones in the red channel meaning 0 depth) when not having the boundary enabled or very weirdly warped and broken. Since, as far as I know, the depth data is computed using an ai model with the camera data and I think also the spatial data from room scans etc. it could be faulty when the ai model is expecting new data or something like that.
    I was able to figure out that in order for the Depth API to compute correct depth information the projekt now NEEDS to be set up to use passthrough. That does not mean that the passthrough needs to be used but it needs be enabled and that fixes everything. For that the OVR Passthrough Layer script needs to be added and the enable passthrough must be checked in the OVR Manager. I don't want to display the passthrough so I just set the oppacity to 0 and everything works like before.
    Maybe it's related to that and my "fixes" can help but maybe it's another bug introduced by meta in the v72 update...

    Aethoen
    New Participant
    February 10, 2025

    I was able to get the DepthAPI working again using an Android build running on-headset, but still no luck with using the Unity Editor. Hopefully they can fix it soon, I don't really want to keep saving and transferring files off the Quest 3...

    Terms & ConditionsAccessibility statement