Shadow Quality/Stability Issues

To bump my own thread here, from playing around, it looks as though the output of the shadow buffer isn't being placed onto the scene properly, or something is wrong angularly. Note the following screenshots:

shadowDrift.jpg

Looking for theories as to why this is happening.

Its caused by the custom projection matrix. It breaks Unity's ability to project back into world space to sample the shadow buffer.

Modifying the camera class to move the camera transform instead of applying the offset to the matrix fixes it, but its possible that could cause other visual differences. I don't have a rift to test with yet.

@owenwp: Thanks for the insight! Is this an issue specific to the oculus, or does Unity's deferred lighting break with all custom camera projection matrices (to your knowledge)?

Either way, this is rather troubling, considering most of the high-end aesthetic approaches in Unity tend to involve deferred lighting...

Its a limitation thats been around for a long time. I ran into it before when I was doing off-axis perspective projection, and I found a solution but I don't remember exactly how I implemented it (old job, can't get the code now). The skybox issue is another symptom of the same problem as well.

I'll see if I can reproduce the fix, but for now you can just remove the custom matrix code and subtract the offset from the eyeoffset vector during camera setup, which should give you the exact same result. I think that may be the better solution in the long term anyway because it allows you to access the actual eye positions in shaders and scripts, instead of having the offsets baked into the perspective transform.

Forgive my ignorance, as I'm not used to directly dealing with camera projection. Is the code in the OVR camera just offsetting the root position? Or is it also rotating/doing an asymmetric frustum for having the eyes converge at distance? ie. would just placing the cameras by hand be _exactly_ the same as their projection setup?

Thanks for the help with all this btw!

They use parallel projection with a centered axis according to the docs, and the code in their camera class looks like its constructing a standard projection matrix mostly consistent with the Direct3D implementation (one value is flipped, this might just be for handedness, Unity probably does the same internally), so yes there should be no difference at all.

To enable deferred lighting I turned it on for CameraLeft and CameraRight then edited the OvrCamera (script) file to add at line 221:

if(camera.actualRenderingPath == RenderingPath.DeferredLighting) {
	flipImage = false;
}

The shadows flicker and shimmer with every head movement and are moving with my motion instead of being fixed with the geometry itself, but at least it isn't flipped.

Did you guys ever find a fix for this? I'm finding a number of shadow problems, which is a real shame because I feel like shadows really add a huge amount to the already great sense of depth.