Skydome + virtual objects (AR scenario) problems

Hi there,

I have the following Unity scene (not considering the Rift yet):

• 1. A skydome, that is, a sphere with inverted normals on which a panorama image is mapped. The diameter does not matter, but for now I've used 2 meters (= 2 Unity units) as an arbitrary value. This sphere represents the "reality" part of my AR application. It is placed in a user layer called "Background" that I created.


• 1.1 A "Main camera" that renders the virtual objects (see point 2 below). Its clear flags is "depth only", and culling mask is setup such that it includes all layers except the background layer. The camera itself (and its child) is in the Default layer


• 1.1.1 A "Background camera" which renders only the skydome image. By assigning depth values intelligently, I enforce the background camera to be rendering to screen first, followed by the Main camera. This way, the virtual objects are always shown, regardless whether they are inside or outside the sphere.


• 2. Several 3d objects that may be inside or outside the sphere (these are the virtual objects that augment reality). They are in the default layer.

The transform component, i.e. x, y and z coordinates, are all set to 0/0/0 for the elements.

Now this works fine when using other HMDs, or simply rendering it to screen. That is, rendering the same image on both eyes.

Now I tried to integrate the Oculus Rift and I'm running into issues. First, I had to create a copy of the CameraLeft (named CameraLeftBG for “background”), likewise for CameraRight (getting a CameraRightBG) and configure their culling mask and clear flags accordingly. Then I modified the OVRCameraController script: I created two additional camera instances, OVRCameraController::CameraLeftBG and OVRCameraController::CameraRightBG, assigned them in Awake(). In UpdateCameras() I added two calls to ConfigureCamera(). In SetPortraitMode() I also added some code (straightforward copy/paste, changing CameraLeft to CameraLeftBG, likewise for CameraRight).

However, I discovered that when testing this, the virtual objects would not stay in fixed position (relating them to the skydome image, that is), but they would be "floating" unexpectedly. This can be mitigated by disabling the "Lens correction" checkbox of the camera controller object, but then I would somehow get headaches when looking at the virtual objects, having the impression that I have to "concentrate" as otherwise I'd feel as if I were squint-eyed. Playing around with the IPD value would improve the feeling, but I'm not able to get rid of it completely.

I understand that it may have to do with the fact that I'm combining a flat picture (skydome) with true 3-d information (virtual objects). But I don't have enough background knowledge to know how to proceed. I would be fine having stereo rendering disabled completely, but I'm not sure how to do that or whether it will help.

For your reference, you can find the example project here. When you disable the OVRCameraController in the hierarchy and enable the MainCamera, you can move the mouse to simulate the headtracker.

Best regards!
Marius