Changing Rift viewport and rendering cameras after the Rift

"vrdaveb" wrote:

The entire Oculus C API is exposed in OVRHmd.cs. There is some code commenting in there, but the full version is in the native SDK's OVR_CAPI.h header.

I can't find a method called ovrHmd_RegisterPostDistortionCallback(..) in either OVRHmd.cs or OVR_CAPI.h. Is this method exposed in the OculusPlugin.dll for Unity?

"vrdaveb" wrote:

OVR_SetViewPort(..) is an internal function that I wouldn't recommend calling from your app. To render additional viewports with good vsync behavior, you probably need to write some Win32 code and set up your own windows and swapchains instead of over-sizing the VR window.

Good point. Forgot the DK2 is running at 75 Hz. Do you have any suggestions on how to get the functionality I described in the original post?

"ryahata" wrote:
Our project has a requirement that we display content in the Rift, and different content to one or more additional displays connected to the same PC.

It sounds like you're having Unity render to one or more cameras in addition to the Rift. One thing you could do is set up an additional IDirect3DSwapChain9 and IDirect3DDevice9 for each additional camera and copy the data from the camera's native texture pointer (see http://docs.unity3d.com/ScriptReference/Texture.GetNativeTexturePtr.html) to the additional devices for display. There are other ways, too, such as running multiple apps and synchronizing them over sockets or shared memory. You should be able to find online resources for that.

Thanks for the quick responses, vrdaveb. Any word on the ovrHmd_RegisterPostDistortionCallback(..) and how to use it?

"vrdaveb" wrote:

One thing you could do is set up an additional IDirect3DSwapChain9 and IDirect3DDevice9 for each additional camera and copy the data from the camera's native texture pointer (see http://docs.unity3d.com/ScriptReference/Texture.GetNativeTexturePtr.html) to the additional devices for display.

How would I make sure that it doesn't interfere with the Rift's rendering? Because as I stated earlier it seems that the black mask is drawn to parts that extend beyond what should be the Rift's viewport.

"vrdaveb" wrote:

There are other ways, too, such as running multiple apps and synchronizing them over sockets or shared memory.

Yeah, that's plan B. Unfortunately it requires bit of an architecture change which, if avoidable, I'd rather not do.

Thanks again for all the help.

"ryahata" wrote:
Any word on the ovrHmd_RegisterPostDistortionCallback(..) and how to use it?

Sorry, that isn't a public API. You'll have to render to a separate swapchain.

"ryahata" wrote:
How would I make sure that it doesn't interfere with the Rift's rendering?

We only clear and render to the main swapchain and backbuffer. If you render to a texture, copy the texture to another device, and display it in a separate swapchain, we won't clear it.

"vrdaveb" wrote:

Sorry, that isn't a public API. You'll have to render to a separate swapchain.

Ok that makes sense. Completely missed the comment over the function declaration. :(

"vrdaveb" wrote:

We only clear and render to the main swapchain and backbuffer.

I'm not really familiar with this lower level stuff so I apologize in advance. It sounds like I have to create my own native render plugin that receives render textures from Unity and then create one swapchain per extra monitor that I want to display to. Since the render threads are separate I should be fine in terms of vsync, correct? Does this all sound about right?

Thanks again vrdaveb for all the help! :D

Yes that sounds about right. You can find example code here for sharing textures with native code: http://docs.unity3d.com/Manual/NativePluginInterface.html. When you pass down a RenderTexture, you should call RenderTexture.Create() before GetNativeTexturePtr() to ensure the native resource exits. From there, you would be locking the texture, copying it to the secondary device, and blitting or rendering a regular textured quad with D3D9.

We face similar problems as already described in this thread: We have created a VR toolkit for Unity, where the developer can configure various multi-display setups with any number of displays, by rendering all the display viewports side-by-side, resulting in a wide game window (Unity application, windowed-mode). With the Oculus Rift this scheme has worked when the Rift's display has extended the desktop, and the Unity game window has been moved by the user to the correct location in the desktop. With the current DK2 SDK it seems that there is no elegant way to do this. Writing native plugins is out of the question, because that would just complicate the developers' understanding how our multi-display scheme works in our VR toolkit (which also supports both Windows and OSX).

For the next version of Oculus SDK, please consider our following requests, which only need to apply when DK2's display uses "Extended Mode":

1. In the Update() method of OVRCameraController.cs you have the following invocation: OVR_SetViewport(0, 0, 1920, 1080). Please modify this to something like OVR_SetViewport(x, y, w, h), where the x, y, w, and h are public variables that can be modified from external C# scripts (we would prefer not to hack any of your OVR scripts, as each transition to a new Oculus SDK would require re-introducing the hacks). Also it seems that altering the first two arguments (x and y) do not have any effect currently, so please fix this so that we have full control of where to render the viewport in Unity's render window.

2. Limit the region where the black mask is drawn (or screen is cleared), so that the areas around the viewport (defined with OVR_SetViewport(x, y, w, h)) are not affected. Currently our multi-display test configuration with Oculus DK2 and a 2D display is rendered incorrectly like this:


We have a standard Unity camera rendering on the right side (which is incorrectly black in the above image), and therefore we should be getting this:


So something in OVR code clears or masks the Unity framebuffer, and no matter what Camera.depth we set, the situation is not fixed.

Interestingly, if I resize Unity's game window while its playing, the camera viewport on the right side comes through for every other frame or so.

We're having the same issues listed above and would also like to make that request if possible.

"Ttakala" wrote:
Please modify this to something like OVR_SetViewport(x, y, w, h), where the x, y, w, and h are public variables that can be modified from external C# scripts

Sorry, OVR_SetViewport is not a public API. It is there to communicate internal rendering information from Unity to our native plugin, not to support additional non-VR viewports or arbitrary positions.

"Ttakala" wrote:
So something in OVR code clears or masks the Unity framebuffer

In 0.4.0 and onwards, we always clear the entire backbuffer. Under the hood, D3D 11 does the same thing, so it would be hard to make it work another way across all the platforms we are targeting.

"Ttakala" wrote:
Writing native plugins is out of the question, because that would just complicate the developers' understanding how our multi-display scheme works in our VR toolkit (which also supports both Windows and OSX).

Unfortunately, this may be your only option for 0.4. There are a number of reasons not to share a single backbuffer between the Rift and other monitors. First, DK2 refreshes at 75Hz. If the rest of the user's monitors use different frequencies, there will be judder and tearing. Second, moving forward, direct mode will produce the best experience in most cases. Today, you could continue using 0.3.2, run all your displays at the same frequency, and span as you've been doing. But direct mode will soon offer significantly lower latency and smoothness (via asynchronous timewarp) compared to extended mode. That being said, I will raise this to the team and see if there's something we can do to support use cases like this in the future.

Thanks for your reply vrdaveb!

"vrdaveb" wrote:
Today, you could continue using 0.3.2, run all your displays at the same frequency, and span as you've been doing.

We have a DK2 and we need to access the head position within Unity.

"vrdaveb" wrote:
There are a number of reasons not to share a single backbuffer between the Rift and other monitors. First, DK2 refreshes at 75Hz. If the rest of the user's monitors use different frequencies, there will be judder and tearing. Second, moving forward, direct mode will produce the best experience in most cases.

I agree that Oculus' Direct Mode is way to go in most cases. I'm pleading for that in the future developers would have the option to use extended mode to render additional viewports besides the Oculus ones, even with the cost of vsync tearing and additional latency. Using such approach, our students created some pretty amazing VR experiences with DK1, and we'd like to keep doing that in the future with DK2 as well:

Wheelchair Champion


Virtual Curling


The above examples were created in our VR toolkit for Unity, using one wide game window with multiple viewports. Certainly the demonstrator with Oculus experienced more latency due to the additional view rendering, but otherwise the audience of our demo event could not have enjoyed the game (without multiple Unity instances and networking between them, which is too much to ask from most students). Furthermore, with our toolkit it's easy to drop the additional "audience viewports", and render only the Rift's view.

"vrdaveb" wrote:
That being said, I will raise this to the team and see if there's something we can do to support use cases like this in the future.

Much appreciated!