New Participant

Quest3 and UE: Virtual Window/Portal through passthrough into a virtual world

5 replies
3110 views

Hello everyone,

We're still struggling to get a "window/portal into a virtual world through passthrough" effect working on the Quest 3 with the latest Meta Plugin and Unreal Engine 5.3.

There does not seem to be a working "inverse" Punch-a-hole material that punches a hole through an overlay passthrough layer, which would be basically ideal.
Alternatively, I tried to use an underlay passthrough layer and get a regular portal effect via custom depth & stencils, but that just does absolutely weird things when deploying on the Quest itself, I assume because post processing / mobile deferred rendering is really not well supported.

Does anyone have any other pointers into what could work? I kind of just want my real window/door look out into a virtual word. We'd be really thankful for any help!

This topic has been closed for replies.

CarpetFace

Inspiring

I made a separate post expressing my own desire to do something like this and expressed the limitations of the reprojection artifacts I ran into. I'm not sure how well this entire scenario is supported on UE right now, so I am waiting for it to mature a little more myself. However, if I desperately wanted to get it working right now, what I would try next is the following, since I've observed reconstructed to be the highest quality with the least amount of reprojection artifacts, so I would basically reimplement a version of that.

First, get the room data geometry, the same geometry that reconstructed pass through is using. Next, where ever you want your game world to show up, remove holes in that geometry, and then finally, use that geometry via AddSurfaceGeometry and overlay mode.

V

vrgroupAachenAuthor

New Participant

Hi! I saw your post as well! I'm not quite sure if I'm even able to directly access the geometry of the reconstructed scene, at a first glance that is locked within the Meta binaries.
I'll take a further look and see if the mesh is modifiable - even if not the vertex shader / world position offset might be an okay hack. Thanks for the tip!

D

dariomaciel

Honored Contributor

I did with custom depth and stencil and it worked for me. I could only see a virtual mesh from within a invisble plane in front of it. Is that what you are looking for?

V

vrgroupAachenAuthor

New Participant

Hi! I also tried it with custom depth and stencil, and while it works in Unreal itself, I didn't get it to work on the Quest itself. Did you do it in a post process material, or how did you access the stencil buffer to test?