How to do Multilayered Cameras With 0.4

Thanks a lot for this sub-camera script. However, I'm experiencing that strange Editor-only issue where the rotation seems to be doubled up as in some other posts. The scene is rotated according to the Rift's orientation and rendered into the render textures, then those distorted render textures are further rotated by the same amount (if you keep rotating, the two halves rotate fully out of the editor play window).

I've triple-checked that I don't have a second OVRCameraController or anything else weird. I also tried removing all the OVRSubCamera stuff and am still seeing this problem.

A standalone build works correctly. This is all in DX9 mode. Thanks in advance for any help.

EDIT: Today it has been working fine in the editor from the first attempt. I can't explain why, but I'll take it!

"vrdaveb" wrote:
Here is a similar script that also takes care of the projection, target texture, and a few other things. To use this, just set up the OVRCameraController and OVRCamera rig for your near cameras and make a "sub-camera" for each of your far-cameras.

This looks super useful, but I'm having issues getting the two sets of cameras to sync up perfectly. In a standalone build (possibly in Editor, too, but the frame rate is too high to tell), it's apparent that one of the camera pairs is slightly lagging the other. I tried some basic Unity things like switching the sub camera's Update to LateUpdate and changing script execution order, but I don't think these will work because OVRCamera does so much in OnPreCull. I also tried reversing the link: I put sub-camera references in OVRCamera itself, and converted Start and Update in OVRSubCamera into Init and SetRotation methods that I call from OVRCamera. I've also tried putting the OVRCameraController on either the exterior set or on the interior set, but both setups exhibit the problem. No matter what I do, there's still some tiny amount of lag, like the exterior is swimming relative to my interior.

I'd love to continue using a multi-camera-pair setup for a couple of reasons, so hopefully there's some simple trick that will eliminate the synchronization issue.

[Moved response to different thread]

Thanks for the script vrdaveb!

What do you suggest if we want to render UI cameras on top of the world cameras? I find it easiest to just put the UI onto its own camera layer so that I can guarantee that it renders on top of the world without having to muck around forcing depth in shaders or using render queues.

Your script actually mostly works for this purpose (even with the UI cameras rendering afterwards) but I do end up with whatever the left camera rendered doubled up on the right camera, which obviously isn't ideal.

"excelynx" wrote:

Your script actually mostly works for this purpose (even with the UI cameras rendering afterwards) but I do end up with whatever the left camera rendered doubled up on the right camera, which obviously isn't ideal.

This was the problem I had with 0.3 (duplicate in one eye) I thought this was fixed with this new solution?

"LaeusVR" wrote:
I'm experiencing that strange Editor-only issue where the rotation seems to be doubled up as in some other posts. The scene is rotated according to the Rift's orientation and rendered into the render textures, then those distorted render textures are further rotated by the same amount (if you keep rotating, the two halves rotate fully out of the editor play window).

This sounds like a thread ordering issue with our TimeWarp usage. There will be a release soon that should fix this. For now, does it help to turn off OVRCameraController.TimeWarp?

"LaeusVR" wrote:
In a standalone build (possibly in Editor, too, but the frame rate is too high to tell), it's apparent that one of the camera pairs is slightly lagging the other.

Sorry, the problem here is that the far cameras are rendering with pose data from the previous frame. Can you try swapping the near and far camera pairs? This would mean the OVRCameraController AND OVRCameras would render the far image and the OVRSubCamera would render the near image afterward. You'll have to swap Camera.depth between the near and far rigs.

"excelynx" wrote:
What do you suggest if we want to render UI cameras on top of the world cameras?

If you are using OnGUI, you should check out OVRGUI. It lets you render standard Unity GUI elements to a texture. Unfortunately, there is no support for mouse interaction. I would recommend NGUI or NoesisGUI for anything more involved. Or you could wait for the upcoming uGUI.

"excelynx" wrote:
I do end up with whatever the left camera rendered doubled up on the right camera, which obviously isn't ideal.

Are you using any image effects? There is a known issue where Unity will drop the far camera's output when image effects are used.

"vrdaveb" wrote:
If you are using OnGUI, you should check out OVRGUI. It lets you render standard Unity GUI elements to a texture. Unfortunately, there is no support for mouse interaction. I would recommend NGUI or NoesisGUI for anything more involved. Or you could wait for the upcoming uGUI.

Are you using any image effects? There is a known issue where Unity will drop the far camera's output when image effects are used.

I'm currently using NGUI which is floating in world space - thus the need to force it to render over the world geometry. I'm definitely looking forward to getting my hands on the new Unity GUI.

I am using Image Effects - but I'm trying to use this script to work around that issue: http://forum.unity3d.com/threads/deferred-renderer-multiple-cameras-posts-ssao-bugged-ufps-onrenderimage.198632/#post-1348581

Edit: While the CopyToScreenRT script works fine with a single camera stack, it appears that it causes bugs with the Oculus dual camera setup anyway (although only with the right camera copying over the left camera when using OVRSubCamera) so I'm trying to find a better solution or a workaround for that problem as well.

"vrdaveb" wrote:

"LaeusVR" wrote:
(...) issue where the rotation seems to be doubled up

This sounds like a thread ordering issue with our TimeWarp usage. There will be a release soon that should fix this. For now, does it help to turn off OVRCameraController.TimeWarp?

Yes, this does appear to fix it. I also had to restart Unity.

"vrdaveb" wrote:

"LaeusVR" wrote:
(OVRSubCamera) one of the camera pairs is slightly lagging the other.

Sorry, the problem here is that the far cameras are rendering with pose data from the previous frame. Can you try swapping the near and far camera pairs? This would mean the OVRCameraController AND OVRCameras would render the far image and the OVRSubCamera would render the near image afterward. You'll have to swap Camera.depth between the near and far rigs.

Unfortunately, that just reverses which cameras are lagging. The exterior cameras now respond "first" visually, and the interior ones follow suit. I also tried disabling TimeWarp (no effect), and then enabled DX11 (didn't fix the swimming, but as in other cases, it cleared up the double vision / "judder"). I'm on Unity 4.5.2p2 right now, and here are some specifics in case they help:

OVRCameraController object: all default settings, except I've tried with and without Time Warp, and I've set a Follow Orientation to its parent object (though I've also tried clearing that with no improvement).
Camera objects below that: clear to skybox, culling mask to exterior layers, depths 1 and 0, OVRCamera components set with Right Eye as appropriate.

Separate GameObject (not in hierarchy with the OVRCameraController), has two other cameras on it: clear to depth, culling mask to interior layers, depths 3 and 2, OVRSubCamera components with Parent fields set to the previous OVRCameras.

The thing that's really confusing me is that I forced OVRCamera to set localRotation on the sub cameras immediately after SetCameraOrientation in OVRCamera.OnPreCull, hoping that this would provide them with the most updated setup. I verified that all four cameras received the same transform.localRotation at the same time. Somehow, that doesn't address the issue. Creepy :). Maybe something is going wrong with retrieval/setting of the projection matrix? As an experiment, I tried setting all near/far clip planes to the same value since those go into the OVRDevice.GetCameraProjection call, but that didn't fix it either.

I'm also getting an odd bug with this script where the left eye is rendering as the right eye. This is quite possibly related to the depth ordering I'm using (to get the UI rendered in front) and/or the double rendering in the right eye that I'm getting.

Basically the left eye is rendering in the correct place (on the left side of the screen) but it's from the perspective of the right eye. My parent cameras are both set correctly.

Just an update on multilayered cameras: I've finally tracked down the problem and have sub-cameras working without any latency.

TLDR: in vrdaveb's OVRSubCamera script, replace "void Update()" with "void OnPreCull()". You also have to ensure that your OVRCameras render before any OVRSubCameras by giving them the lowest depth values.

This works because OnPreCull is called after all the Update functions, which explains why none of the other orderings worked, even with manual setting of rotation from OVRCamera -- the sub cameras had already gone through rendering. Some useful info in the Unity docs here: http://docs.unity3d.com/Manual/ExecutionOrder.html

Also, this probably won't work correctly with TimeWarp enabled, per vrdaveb's note that the sub-cameras should be rendering first. Cross that bridge later. TimeWarp makes my builds unusable anyway, so not a big problem for now. :)