cancel
Showing results for 
Search instead for 
Did you mean: 

Suggestion: Allow apps to request permission to access camera feed(s)

Sparkett3
Protege

The developer blog says:

"We built Passthrough API with privacy in mind. Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device."

 

That's all well and good, but what if an application has a legitimate reason to need access to that data? I can think of several such use cases:

  • A camera app for taking pictures/recording video in 3D
  • Mixed reality experiences that implement computer vision algorithms to react to things in the real world
  • A more advanced version of Waltz of the Wizard's mixed reality sandbox that allows changes to be made to objects in the real world (well, at least how they're displayed in the headset 😉) using image processing filters
  • Trippy effects that go beyond the pre-existing filters in the SDK
  • I'm sure there are many others as well.

From my understanding, none of these use cases are currently possible on the Quest, even though there's no reason they shouldn't be with the hardware that's present. I couldn't even code it for my own experimentation unless I find some way to root the headset, which is annoying as I spent $1,500 on a Quest Pro and shouldn't be prevented from using its hardware as I please.

 

So what I recommend is adding an API for accessing this raw image data, which would of course only work if the user grants permission to the app. Apps can already do this on smartphones with no issues, so I don't see how the Quest would be any different. The Oculus Store, of course, can also set policies as necessary, such as requiring apps to keep the image data on device whenever possible, and to never send it to a remote server without the user explicitly giving permission.

 

I'd have posted this in the SDK Feedback section, but for some reason it says I'm not allowed to start a thread there, so I'm posting it here instead.

16 REPLIES 16

Ciberspy
Explorer

Literally 2% chance this will happen this decade.

Why do you say that?

Darkever87
Honored Guest

I totally agree with that. Please make it happen Meta (at least for the pro headset for enterprise usage). MR application is very limited without the ability to recognize objects in user's surrounding. 

kkiieerraann
Honored Guest

its almost like they dont want us to develop XR apps its so frustrating. everything is so awkward for no real reason 

Turisas
Explorer

Yes we need acces to feed from cameras. How can I build an app similar to remote assist apps if I can't get video stream and send it to expert?! Smth like MS did with Hololens and  Dynamic 365 Remote Assist

gil.clayton.14
Explorer

These Privacy Restrictions go too far at limiting the developers! 

I agree, there should be an option to allow saving the images!

I agree, as a developer who wants to run my own computer vision code, I don't see why this roadblock is imposed. If it's purely a security concern, well the passthrough cameras are already viewing my room to perform SLAM on device, the microphone is recording, but asks for permission, why can't the camera's do the same.

Sure its nice to have passthrough via OpenXR, but just compositing content onto a camera feed is pretty useless. As per above, this is putting serious roadblocks in place for AR/Mixed reality development. 

user_was_taken
Explorer

It's hard to believe what I am reading.  This is crazy.  How are we supposed to be able to make XR apps without being able to process data about the environment the device is in? 

What possible reason could there be for not allowing users to opt in?

 

Abuse of privacy, and leak of RAW images.  Now sure how anyone is somehow missing the big elephant in the room; you can't just dimiss the privacy ramifications with a 'Allow Camera Access" opt in boilerplate message.....