"We built Passthrough API with privacy in mind. Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device."
That's all well and good, but what if an application has a legitimate reason to need access to that data? I can think of several such use cases:
A camera app for taking pictures/recording video in 3D
Mixed reality experiences that implement computer vision algorithms to react to things in the real world
A more advanced version of Waltz of theWizard's mixed reality sandbox that allows changes to be made to objects in the real world (well, at least how they're displayed in the headset 😉) using image processing filters
Trippy effects that go beyond the pre-existing filters in the SDK
I'm sure there are many others as well.
From my understanding, none of these use cases are currently possible on the Quest, even though there's no reason they shouldn't be with the hardware that's present. I couldn't even code it for my own experimentation unless I find some way to root the headset, which is annoying as I spent $1,500 on a Quest Pro and shouldn't be prevented from using its hardware as I please.
So what I recommend is adding an API for accessing this raw image data, which would of course only work if the user grants permission to the app. Apps can already do this on smartphones with no issues, so I don't see how the Quest would be any different. The Oculus Store, of course, can also set policies as necessary, such as requiring apps to keep the image data on device whenever possible, and to never send it to a remote server without the user explicitly giving permission.
I'd have posted this in the SDK Feedback section, but for some reason it says I'm not allowed to start a thread there, so I'm posting it here instead.
I totally agree with that. Please make it happen Meta (at least for the pro headset for enterprise usage). MR application is very limited without the ability to recognize objects in user's surrounding.
