cancel
Showing results for 
Search instead for 
Did you mean: 
stellarluke
Explorer
Status: New Idea

I discovered that while using the Meta Quest 2 if I open the Meta Quest app and stream to my iPhone, I can use the built-in Apple screen reader called VoiceOver to hear items under my finger as I touch the iPhone screen.
This could be a simple and effective way to bring screen reader functionality to Meta Quest for visually impaired users.
With the Quest Menu displayed in front of me showing my App Library, I was able to hear the app named "Vader Encounter" under my finger as I touched it on the iPhone. I then double-tapped it and was able to launch the app on the Meta Quest from my iPhone using the screen reader.
However, I had difficulty repeating this and using the screen reader functionality because it was not allowing me to consistently touch the item in the UI. Instead, I heard it saying things like "tree", "sunset", "folliage" and other object names. I tried closing and re-opening the menu but still could not get the VoiceOver screen reader to "focus" on it to speak items I touched.
My idea is to refine this a bit to allow "touching" objects in the Meta Quest environment more selectively. An example would be a setting under Accessibility Vision under the heading "Streaming to Mobile Device". Under that, have an on/off toggle with the label "Limit touches to User Interface". Help text under that could read "When on, touching the mobile device screen or navigating it with a screen reader will be limited to menus and buttons."
Allowing this setting to be "off" could allow the user to also touch and have items in the 3D environment spoken. 

In my experience developing apps in a 3D environment (Unity), I think it is possible to control which selected items in the environment respond. to being touched, and what messages are sent to the gesture handler in iOS. I would think the UI elements would be in a tagged layer. For VoiceOver in iOS, it is able to focus on any items in the app UI that have the "isAccessible" boolean set to TRUE. So evidently the items I am hearding have that value set, and it should be just a matter of selectively setting non-UI elements to FALSE.

2 Comments
Anonymous
Not applicable

Thank you for your feedback!

leang.25557
Honored Guest