Skip to main content
New Participant
May 22, 2024

DepthAPI Occlusion Lit conflict with stencil shader

  • May 22, 2024
  • 8 replies
  • 2448 views

Hi there,

I'm currently working on creating a local multiplayer XR experience that involves both portal effects (using a custom stencil shader) and DepthAPI to allow players to see each other in a shared space.I've managed to get DepthAPI and the stencil shader to work individually, but I can't figure out how to combine them.

The main issue is that DepthAPI
, which uses the depth sensor of the Meta Quest 3, recognizes physical walls around players and prevents pixels of objects that are virtually further from the headset than the walls from being drawn. So, while the portal stencil works, it can't render the "outside" of the room.

Do you have any ideas on how to deal with this
? I'm not skilled enough in shaders to resolve it on my own.
Here some screenshots of my current progress : 


This topic has been closed for replies.

8 replies

jtriveri
Honored Contributor
May 22, 2024

There’s no reason why you need to use the occlusion shader on the environment through the portal. You could just occlude only the stencil surface. If a stencil fragment is occluded, so is everything behind it

 

edit: this may not work with soft occlusion and it may not look right with the right-eye depth reprojection. I’m playing with a similar issue and will get back to you if my other solution works out

New Participant
May 23, 2024

Well I tried playing with the EnvironmentDepthBias to try modify the occlusion z distance but its not working very well. I also tried occluding only the portal w/ stencil shader and here what happens : 
- plane w/ stencil is occluded so there's a "mask" arround hands 
- through this mask, external environment is drawn over hands so hands aren't visible 
- same with every real world furniture that should be drawn over the environment

jtriveri
Honored Contributor
May 23, 2024

What does your stencil portal shader look like? Can you paste it here?

New Participant
May 28, 2024

Here it is, its really basic : 

Shader "Custom/StencilPassthroughShader"
{
    Properties
    {
        [IntRange] _StencilID("Stencil ID", Range(0, 255)) = 0
    }
    SubShader
    {
        Tags
        {
            "RenderType" = "Opaque"
            "Queue" = "Geometry"
            "RenderPipeline" = "UniversalPipeline"
        }
        Pass
        {
            Blend Zero One
            ZWrite Off
            
            Stencil
            {
                Ref[_StencilID]
                Comp Always
                Pass Replace
            }
        }
    }
}
New Participant
January 20, 2025

Did you ever find a solution? I'm dealing with a similar challenge.

New Participant
March 26, 2025

I have setup my stencil but its not working too.

Sandstedt
Inspiring
April 15, 2025

You can add your meshfilter to the Environment Depth Manager's Mask Mesh Filters, and it will remove it from the depth map. Works for my stencil shaders, at least. It doesn't look the best when using soft occlusion. But it works at least. I have also set the shader to user Environmental Depth Bias -0.07. Worked best for me.

 

Unity XR Developer
Daemyn
Participant
June 18, 2025

Do you mind if I ask how you got the stencil shader to occlude? I currently am working on a similar setup, and while the depth occlusion is working great on objects with an occlusion shader, it is not working on the stencil as it is (which is much the same as yours), and it sounds like you have it working with occlusion without needing to modify the shader?

As to your original question, as Sandstedt pointed out, the Depth Mask Mesh Filters is probably your best bet. You can also add the meshes to the depth mask list automatically with code, shown here: https://developers.meta.com/horizon/documentation/unity/unity-depthapi-occlusions-advanced-usage#using-the-depth-mask-feature (before I got started, I knew I would have a similar issue, and that's what came up)