A messenger for exchanging passthrough modes between two Oculus Quest 3 headsets
Good day, everyone! I'm working on a project aimed at implementing virtual body swapping using standalone VR headsets. I want to stream the video from the see-through cameras from one headset to another. The problem lies in API access restrictions and processing video with depth data so it looks like the headset's native see-through channel. Ideally, the system application that renders the see-through image captures the camera video and the LIDAR data from the headset. We cannot directly transmit the finished image to another headset. If we use the API, the image will be cropped, according to the documentation, and we'd need to write the rendering algorithm ourselves, as simple flat stereo video doesn't provide immersion.
My idea is this: we can transmit the see-through camera and LIDAR data over the network to another headset and trick the second headset's system application into capturing this data from the other headset instead of its own, processing it the same way it processes its own data, and creating a full-fledged see-through mode based on this data. Is it possible at all for a developer to gain access to this system application to adapt it for the messenger's tasks?
