Skip to main content
SparkyTD
New Participant
March 15, 2020

Is it possible to access the passthrough camera feed programmatically on the Rift S?

I know that the feeds are only b/w and they are full of random artifacts, but it would still be convenient if I could use them for an OpenCV program running in my VR app. Is this possible with the Oculus SDK?

    8 replies

    MikeF
    Inspiring
    March 16, 2020
    no, asked 100 times over. not going to happen.
    JeffNik
    JeffNik
    Honored Contributor
    March 16, 2020

    MikeF said:

    no, asked 100 times over. not going to happen.


    @MikeF - Why are so confident that they wont eventually make it available?
    Jame1s
    New Participant
    April 2, 2020
    Is there a way to turn on the pass through camera for a bit to reorient myself in my room or to grab something really quickly?
    SparkyTDAuthor
    New Participant
    April 2, 2020

    Jame1s said:

    Is there a way to turn on the pass through camera for a bit to reorient myself in my room or to grab something really quickly?

    @Jame1s On the Rift S, you can double press the menu button on the right controller and it will do exactly that. I'm not sure about other HMDs.
    JeffNik
    JeffNik
    Honored Contributor
    April 2, 2020
    @Jame1s, @SparkyTD: Note that, in version 14 at least, that feature is off by default - you have to go to settings in the headset, under Experiments, and set "Passthrough Shortcut" to "on".

    Quest is supposed to have passthrough on demand in version 15, (experimental setting, too), and its toggled by tapping on the headset itself twice. I'm very much looking forward to that update!
    MatrixPaul
    Participant
    January 15, 2021
    I would also like to access the pass-through camera output for Unity or indeed any other way. Perhaps even just taking a screenshot of the scene with passthrough enabled.
      noshame1232
      New Participant
      April 23, 2021

      Hey, any update on whether you were able to use passthrough with unity? I am currently working on a project that would greatly benefit from this. It won't stream to my phone too. Also, I have the oculus quest 2 so not sure if there have been any recent updates that may have changed this. 

      pabramsor
      pabramsor
      New Participant
      May 29, 2022

      Now it is possible to enable passthrough in Unity, there is an excellent tutorial on the official webpage: https://developer.oculus.com/documentation/unity/unity-passthrough/?locale=es_ES

       

      But for now (and it seems that it will be forever), the only thing you can do is underlay or overlay the camera projection. The images cannot be touched/accessed from the code. It is neither possible to post-process them because the overlay/underlay are combined in a safe and separate stage by the compositor (As far as I understood). At least for the Quest 2 devices. 

       

      What I am not 100% sure about is, and it is the reason why I reached this post, if it is possible to access/postprocess the images using the Oculus Rift S. I did it with Valve Index headset using SteamVR and OpenVR SDK. Oculus Rift is stated to be compatible with SteamVR, but I don't know if the passthrough is hindered somewhere in the low-level driver.

       

      Any update on this by mid-2022?

       

      UPDATE:

      Sadly, it seems that passthrough won't work on Oculus Rift https://forums.oculusvr.com/t5/Unity-VR-Development/When-will-the-passthrough-v31-SDK-be-available-for-Unity/m-p/880516/highlight/true#M19770

      Terms & ConditionsAccessibility statement