Passthrough camera data is not available on Quest 3 developing with Unity

I totally agree. It is a mandatory to perform mixed reality application and synchronize virtual object with real ones = Augmented Reality !

I absolutely agree

Here is an interesting post on the need to open VR camera access to developers and possible ways companies could provide this while still focusing on privacy. In the post it is mentioned that Pico 4 Enterprise headset provides camera access if requested, which is an interesting solution that Meta could use with their headsets - https://skarredghost.com/2024/03/20/camera-access-mixed-reality/

---- Update on this topic ----

About a week ago, Meta announced a research that Reality Labs have been working for some time now. This research is probably the most interesting one since the launch of the product, cause it focuses on user's space understanding by using AI to be used (what I presume) within an API.

This allow the developers to access a layer of consciousness about the user's sorroundings. 

Nevertheless, this does NOT solve the main problem presented at this topic, which is the lack of raw data camera access. In addition to that, Meta is showing zero interest in replying to our needs.

I wouldn't expect that API to be released in the near future. Although it it a step forwards computer vision applications, it doesn't solve our needs.

That's all for today. Thank you everyone for the support and please spread it as much as you can so we can finally have the feature.

Have a nice day.

Ginés.

I saw some people posting here about using the camera feed to help users with vision accessibility, here is an interesting device called Lumen Glasses working on this - https://www.dotlumen.com/glasses

I know the Lumen Glasses device is dedicated towards accessibility use and probably has a very good LiDAR or ToF sensor, but, hopefully, this will be another example that shows the importance of having access to raw sensor data and nudge Meta to allow developers access to the camera feed.

i'm trying to research a way to access camera image data, i found some usefull info that might make ways for an unofficial access to camera data.  https://www.reddit.com/r/OculusQuest/comments/1c1s2ae/part_2_of_trying_to_find_an_unofficial_way_for/

Here is another interesting use for camera access, AR surgeries. Here the surgeon used the Quest 3 but also mentioned his system could be used on other AR headsets like the Apple Vision Pro - https://www.levita.com/blog/levita-magnetics-leads-the-future-of-surgery-with-augmented-reality-ar-headset-enhanced-abdominal-surgery

Unfortunately, in the press release above they do not go into any detail on what exactly the AR system entails (i.e. are they tracking something in the person they are performing the surgery on or is it, most probably, just a heads-up display with real-time vital signs and body scan information to orient the surgeon). Though this example most likely isn't true AR (which requires the mix of real and virtual objects, lighting, shadows, occlusion; interactivity, physics, object manipulation; and is registered/anchored in a 3D position so you can walk around the virtual object), it does show a small glimpse of how surgeries could be improved.

It's strange camera be inacessible since there are no proper reason to that, even privacy isn't enought, since can do like modern phones and ask camera permission and shown some type of icon when recording.
Some years back android phones aren't able to record the screen and the community did the dirt work with root, nowdays root is pretty useless for normal people since major features that are really usefull the stock phones included into it.
For me lock the system for nonsense things like that just made risk in the end of day, the community may try to do what be needed to exploit the device and that may cause users that just want acess to some basic features have an device with security broken due some dumb missing feature, well, that scenario will happen only if the user base increase a lot and meta keep the basic features locked creating a reason for devs find a way to exploit the system, that's why I think this type of thing to be a very bad decision in all ways.

He says on his Linkedin post; "I’d just cast the quest pro to my MacBook, create a webpage for my MacBook that captures the screen, and send still images of the screen capture back to the quest pro via websockets". Come'un guys, dig a little bit more: https://www.linkedin.com/posts/zakaton_ar-object-detection-on-the-quest-pro-using-activity-7097050766925844480-e9Tl Probably use something like this: https://stackoverflow.com/a/43560140

Hello!
Any updates on this?
I am a PhD student starting a new project and would need to share the passthrough image between multiple headsets for a collaborative task.
I would have guessed this would have been an updated feature by now but I cannot seem to find anything about it. Thanks!