07-21-2021 11:17 AM - edited 07-22-2021 01:48 PM
These features and enhancements will become available starting the week of July 20, 2021.
07-21-2021 11:52 AM
I am really looking forward to this update. As noted over on the Quest forum, this should come with the experimental feature that allows developers to build software to the Mixed Reality API:
Hopefully we can start to see more VR Experiences taking advantage of the Quest's passthrough camera. Fun stuff!
07-21-2021 12:01 PM
Good stuff! Looking forward to testing the AirLink improvements and other QoL enhancements when the update eventually rolls out to my headset.
i7 5820K @ 4.25GHz | RTX 3080 Ti FTW3 | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
SSDs: Intel 660p M.2 2TB, 3x Samsung Evo 1TB | Startech PCIe 4x USB 3.0 | Startech PCIe 2x USB C 3.1 gen2
07-23-2021 11:33 AM
@Zenbane wrote:
this should come with the experimental feature that allows developers to build software to the Mixed Reality API
The question is how long after users get the update will developers get an SDK or docs? Currently the mobile SDK is still v29 and the Rift SDK is v23 from last year. (Not that I expect the Rift-S to get any form of passthrough api)
07-23-2021 11:40 AM
Passthrough API info on what it can do:
https://developer.oculus.com/blog/mixed-reality-with-passthrough/
As expected, no access to the cameras is provided (for privacy reasons) directly to developers, so we can't do things like fiducial marker tracking for fixed world tracking spaces. But it does allow you to render over the top of a live view.
07-23-2021 11:45 AM
I was wondering if Facebook had changed their stance on the privacy issues around camera access. Anyway, I would assume that this will allow them to do things like Infinite Office's passthrough windows to see your keyboard or desktop?
i7 5820K @ 4.25GHz | RTX 3080 Ti FTW3 | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
SSDs: Intel 660p M.2 2TB, 3x Samsung Evo 1TB | Startech PCIe 4x USB 3.0 | Startech PCIe 2x USB C 3.1 gen2
07-23-2021 12:09 PM
I hadn't considered the privacy issue, but admittedly, my enthusiasm was initially focused on the interacting and overlapping of the virtual world with the real world. But yeah, I hadn't consider all the privacy issues this would raise. I wonder how other competitors will handle this situation as we march forward in to the Mixed Reality future.
07-23-2021 02:52 PM
Looking forward to the update.
To be honest, I'm not looking forward to a Mixed reality future, except in the professional arena where having data in eyesight would benefit.
I have enough apps and games in my backlog to keep me busy indefinitely, but I fear all VR may suffer if MR becomes the next fad. I'm not sure VR would survive the fracture of buyers that switch their habits to buying MR gear and games. I truly believe 3D's decline was greatly influenced by people like me that seen VR as the next step in immersive gaming. I'd hate to see VR loose out to MR
Maybe I lack the imagination to envision it, but I can't think of anything I'd like to play with my room in the background. I've seen my room. Now if it were to generate virtual worlds based on my room setup, I'm in.
07-23-2021 07:31 PM
One of the conspiracy theories around Facebook has been that they can see everything your headset sees (so they could target ads based on posters on your wall, Mark can watch you playing VR porn, etc). But none of the camera view is sent. If any developer can access it, they could sneak in a capture that gets uploaded. Maybe malware authors would start making fake games (not released on the oculus store) for blackmail purposes. Of course they could have a phone style permission approval thing ("This app is requesting camera access"), but it's still risky.
For how other people do it:
Windows MR : same as Oculus, no dev access. Oh, and their passthrough "flashlight" is garbage compared to the Oculus full image one.
HTC : do whatever you want. Devs can access the front cameras on the Vive.
Zed Mini : (adds colour and depth passthrough for AR on a VR headset) full access with depth mapped room reconstruction and live stereo image capture (you can use it as a webcam).
Leap Motion : if you don't disable analytics in the settings, it can upload low res images of what it sees to the creators (I believe it's a 32x32 short range grey scale).
I'm not sure about the Index, but it probably follows the Vive.
07-23-2021 07:42 PM
@ohgrant wrote:
Maybe I lack the imagination to envision it, but I can't think of anything I'd like to play with my room in the background. I've seen my room. Now if it were to generate virtual worlds based on my room setup, I'm in.
Imagine playing a game like beat saber. The ground area doesn't really matter that much, except to stay centred. You have a dog or cat (or child) that likes to move around you when you are in VR. With this passthrough system, you could have just the floor region fade to passthrough near you, leaving everything above as the game. So now you can see if your cat/dog/child runs up to you before you step on them.
Or playing a flight simulator where you can see your hands on your keyboard and hotas, but still in the game. This would be especially good for people who build their own cockpits, have the lower half be passthrough (to see where all their custom physical controls are) but top half is the canopy view of the VR world.
It's not going to be Magic Leap / Zed Mini style AR worlds (cameras are low res and grey), more of a safety / productivity feature.