Native Mesh and Depth API
Hi There, Yesterday Meta released two new APIs in the form of Unity and Unreal plugins for accessing depth map and scene mesh data from the Quest 3, however yet again I feel native developers have been left in the dark. I also know WebXR has these features given the talk at connect this year. However, what about people developing their own engines or games natively for VR/MR on Quest ? I'm not sure if it's possible to access the scene mesh via XR_FB_triangle_mesh in OpenXR ? But certainly it doesn't look like any depth extension is present. Please can we have feature parity for OpenXR, or at least release the APIs for native developers. I'm not sure if any devs at Meta read these posts ? Thank you.
External Depth sensor - 3d meshes
I am doing a project where I would like to mount a depth sensor such as a 3D depth camera, sonar or Lidar onto the HMD, to detect obstacles in real time and then represent them as point cloud data or 3D meshes onto the display. This way the wearer can navigate more safely. It is important that I can scan and represent the objects as meshes dynamically in real time and not statically scanned beforehand. Then I would have meshes change colour based on the user's distance from that object. Bear with me as I am clueless on how to start and never used sensors or created an algorithm for the Oculus quest, used unity or anything of the sort. For example would I need an adruino/ rasberry pi to for the depth data or can it be somehow handled by the Quest 2 and using Unity (or unreal idk which would be best to use)? Additionally any recommendations for a cheap depth sensor less than €100? I need it to preferably have a FOV of 90 degrees or better. Remember it needs to go onto the participant's head so needs to be light and compact too. At the end I would like to have a result similar to the below image. However this developer used HTC Vive Pro and its built in depth sensing developer feature to create static meshes and not dynamic. (Link to his paper: https://ir.canterbury.ac.nz/handle/10092/16777 ) Any assistance, material, links and videos that can be provided to get started or confirm this is possible would be very much appreciated.
