Correct way to enable/disable passthrough in app

Using unity 2021.3.35 with the latest Mata API, but I've had the same issues with previous versions of both.

I'm changing if passthrough is enabled with OVRManager in script. If you hover over this setting in the editor it says it can be toggled at runtime, and most of the time it works fine. But occasionally it will cause the app to freeze, forcing you to quit with the home menu. I happens regularly enough to be an issue for users.

If I don't toggle the setting and just hide the passthrough layers it doesn't crash so it must be this that causes it. But at that point passthrough is still on and the GPU and CPU levels are still limited, which gives me performance issues on my non-passthrough environments, so I need this to work.

After reading other posts with the same issue I have tried changing the the setting at RenderPipelineManager_endCameraRendering, so it does it after everything is rendered in that frame, but it has the same result for me.

I'm about to release a big update that uses this, so I need an answer fast. Any ideas?