When will we get object and image classification (Computer Vision) for Quest 3 and Quest Pro?

In every way the Quest 2 and Quest 3 are marketed as a game console product, with Quest 3 even including a voucher for a future released game.  I get that. But you can't do anything meaningful in AR/MR without computer vision. Quest 3 has computer vision, just restricted to common architecture and furnishings. 

Here's an example consumer application that the Quest 3 hardware is capable, but due to the limitations on computer vision can not be built-

Imagine an MR/AR experience where a standard deck of playing cards can be used and the headset uses image detection to determine what's in the players hand (King of Hearts, Seven of Clubs, Ace of Spades, etc...) the experience then teaches the user how to play various card games, spatially anchoring dialogs to specific cards to provide information and clues.

What described above can only be achieved if developers are given access to computer vision models, the company Vuforia provides an SDK and end-to-end solution, where they host the model. As a developer you can upload images for training for your experience. I could build the experience described above using Vuforia SDK and services and make it multi-platform, working on HoloLens, Magic Leap, or on an iPhone or Android phone.  But due to the API restrictions AR SDK vendors like Vuforia and even open tools built on OpenCV will not work with Quest Pro or Quest 3.

With native ARKit development I could build this for Apple's Vision Pro headset. I don't think a lot of people realize it yet, but this means the Vision Pro is going to be capable of delivering this entire broad array of AR type experiences through MR that Quest 3 is incapable of, all due to this restriction and lack of forethought into a CV pipeline. It's kind of assinine when you think about it. 

Without this there is no path to building true MR/AR experiences on Quest 3.  To me, this is the difference between the device being a toy and being taken seriously as a tool. I really hope they provide developers a path to at least perform image recondition. 

---

spacefrog wrote:

Thanks Jeremy  for such an elaborate post about that topic

---

spacefrog wrote:

Thanks Jeremy  for such an elaborate post about that topic
I figure Meta is  just slow shifting their pure consumer focused VR/XR approach over to some more business and technology orientated mindset. Or at least i hope so that this is happening

Consumer VR looks like its staying in its niche for a while longer i think

---

I figure Meta is  just slow shifting their pure consumer focused VR/XR approach over to some more business and technology orientated mindset. Or at least i hope so that this is happening

Consumer VR looks like its staying in its niche for a while longer i think

---

Meta has a big push for AI technologies, which is great. But computer vision like what I'm describing above is the essential AI technology for AR/MR that is entirely off limits to developers.