Giving up on SDK Rendering

I was a bit worried about the sdk rendering but it all worked out well for me. I only used the bare minimum of the OVR sdk, basically only providing the necessary handles needed and generating the matrices but I still add a conversion to the matrices to work with my engine but that not a problem to do. The tiny room sample was great this time round and basically not as much jumping around being in a single cpp filke so found it very easy to just follow step by step. although an individual sample of tiny room for each platform(and version dx9,10,11 etc...) would be better especially when trying to determine if I broke something or the sdk broke something.

I think the main issue I had was that there is no explanation as to what exactly is required to make direct mode work in dx9. I don't know if anything special has to be done, as extended mode direct to rift, works in my engine but I don't know if there is something specific that makes oculus direct mode work, I mean should it just work or not? as this isn't documented at all. I think that was the only frustrating part so far for me. I really like the sdk rendering and saved me a whole bunch of time even though I was going to do it custom as I thought having more control would be desirable, but after using it successfully no way I'd want to use custom.

"lamour42" wrote:
In particular from somebody who is happy with SDK rendering and is using it with his own example app or engine.

Well, I'm using it for examples for my book on Rift development, and I'm reasonably happy with it.

"lamour42" wrote:
Documentation. Some parts are really good. Other part I don't like at all. Worst thing is code snippets. Much too often code snippets are more a documentation to example app structure where it should document SDK API and concepts.

I definitely agree here. There is very little in the way of Javadoc or Doxygen style guidelines for each of the SDK functions, and the samples can't serve as a substitute for that.

"lamour42" wrote:
Example Apps. While I perfectly understand the reasoning to do both DirectX and OpenGL in one and the same example project, it adds to hide the tech details I need to know. Please excuse me egoism, but as a DirectX developer I am not interested in Oculus concepts of dual world programming. I would much prefer examples only for DirectX which then could be simpler. I think I would have the same reasoning if I would be an OpenGL developer.

The examples seem to be hampered by the fact that OVR is applying conventional software development principles (encapsulation, minimal code duplication, modularization) to something that very specifically should not be using them. When you're writing a production app and everyone on the team has a good understanding of the framework, it's a good idea. When you're writing samples it makes them very opaque. That's why I'm writing much simpler examples in my book and working with smaller concepts at each stage. Even the 'Oculus Minimal Example' they provide basically tosses you in the deep end as you have to absorb a whole bunch of concepts at once.

"lamour42" wrote:
Debugging. Using Graphics debugging does not work with SDK mode for me. I get spurious exceptions in my own project when I try to start the graphics debugger. Even the World Demo does not allow for graphics debugging because I cannot capture a screen. (But I didn't see exceptions - obviously I am doing some things wrong in my project).

You've kind of lost me here. Graphics debugging is a pretty nebulous term. I've personally tried using nVidia nSight debugger with the SDK, but it doesn't seem to work. Then again, the inputs to the SDK are just a couple of textures and the associated poses. There shouldn't be a lot of debugging to do across the SDK interface.

"lamour42" wrote:
Architecture. This is the killing argument for me. I failed to get a clean separation between SDK rendering and my engine code. Too much happens under the hood in LibOVR that interferes with my own architecture. Some shaders are set to null, strange things happen with the swap chain, for every frame DirectX complains about shader resource being still bound on output. The list goes on. Maybe if we had a very good documentation about all details that need to be set before calling ovrHmd_EndFrame it would still be ok. But I felt lost trying to find out all the details myself.

This is both a problem that is partially outside the control of the SDK and partly a symptom of their legacy code I think. Rendering APIs have issues with calls that fetch information out of the rendering engine because it typically depends on state that could be modified by rendering commands that have been issued, but not yet executed, so calling any kind of 'get state' call (at least on OpenGL) can cause synchronization points and generate stalls in the rendering pipeline. Most applications deal with this by just having their code either reset a default state every frame or manually keeping track of state changes on the CPU side. But this latter sort of approach can break down when you're dealing with a library that is also potentially making state change.

Looking at OVR's sample code it's clear that they're very much in the first camp (setup all required state at the beginning of every frame) which is probably why they've had so many reported issues with people trying to use their code and complaining about state change leakage.

On the other hand they're aware of the issues and attempting to correct them. For OpenGL, in 0.4.3 they've added code that attempts to create a new OpenGL context specifically for the distortion code, so that any state changes they make are isolated to that context. Unfortunately, this context switching code isn't itself completely bug free, so there's still work to be done.

Very great feedback here, many thanks!

One clarification: With graphics debugging I meant the very powerful capability of Visual Studio 2013 Pro where you start a DirectX app, capture a screen and then can debug all the render code that lead to that screen. This includes rerunning shader code step by step. Also full display of all parameter sets issued with each draw call.

I think that developers who are used to work with that graphics debugger won't accept any framework that interferes with this. It is just too good to loose.

"lamour42" wrote:
One clarification: With graphics debugging I meant the very powerful capability of Visual Studio 2013 Pro where you start a DirectX app, capture a screen and then can debug all the render code that lead to that screen. This includes rerunning shader code step by step. Also full display of all parameter sets issued with each draw call.

That sounds pretty much exactly like nVidia nSight, only that's targeted at nVidia hardware and OpenGL & CUDA apps (it may also support DirectX, I don't know).

"lamour42" wrote:
I think that developers who are used to work with that graphics debugger won't accept any framework that interferes with this. It is just too good to loose.

I hear that, but theoretically you shouldn't have to debug into the Oculus SDK. So for graphics debugging you should render to a non-Rift display. Besides, such debugging probably isn't going to work for 'client side' distortion either, as typically the blocking factor is the injection Oculus does into the rendering stack during ovr_Initialize(), which you still need to call in order to get head tracking.

I think its rather important for this part of the SDK to be opaque. The model for it is the display driver. Displaying an image in VR should be treated as a native render path like fullscreen or windowed, and the particulars of how that works should not be part of the client code, especially since getting any minor detail wrong is catastrophic in terms of latency and the details vary across chipsets, graphics APIs and OSs. This will be expecially important as the technology evolves and we get new and different display technologies using the same interface.

Learning how it all works is an interesting educational exercise, like learning how GLUT handles window management, but it isn't usually a good idea to roll your own. Like GLUT, the Oculus SDK isn't perfect, but the focus should be on identifying and fixing those bugs, not everyone making their own.

I think you would be right in an ideal world.
Ideal world - when it comes to VR rendering - would be a minimal API that does not interfere with the rest of application code. Something like just passing a reference to a texture in graphics memory that should be picked up by the SDK and simply rendered to the device. For many reasons this is not the case. In my opinion, what we have now is an SDK that interferes very much with your application, but doesn't tell us the details about it. That is not opaque, that is information hiding. Always a bad practice in software design.

Compare this to client rendering. Yes, it was unfortunate that I had to throw away all of my DK1 oculus rendering code and start from scratch. But we are in beta and I fully understand that these things are necessary in beta when the devs at oculus try to find out the best approach. Here I get support on a level I can understand and immediately use. E.g. a nice API to provide me with the triangle list for distortion overlay plus shader source code that I can include in whatever engine architecture I have.

It may well be that SDK rendering will be the better choice at some day. Currently, I don't think it is mature enough already.

"lamour42" wrote:
I think you would be right in an ideal world.
Ideal world - when it comes to VR rendering - would be a minimal API that does not interfere with the rest of application code. Something like just passing a reference to a texture in graphics memory that should be picked up by the SDK and simply rendered to the device. For many reasons this is not the case. In my opinion, what we have now is an SDK that interferes very much with your application, but doesn't tell us the details about it. That is not opaque, that is information hiding. Always a bad practice in software design.

Compare this to client rendering. Yes, it was unfortunate that I had to throw away all of my DK1 oculus rendering code and start from scratch. But we are in beta and I fully understand that these things are necessary in beta when the devs at oculus try to find out the best approach. Here I get support on a level I can understand and immediately use. E.g. a nice API to provide me with the triangle list for distortion overlay plus shader source code that I can include in whatever engine architecture I have.

It may well be that SDK rendering will be the better choice at some day. Currently, I don't think it is mature enough already.

It's called Uncle Sam sticking their head in and saying "hey guys, that really cool 4k ultra high-rez stuff? You can't do it- we're using it to dream hack you with spacebots".

The slow-releases keeps the world working for the money machine. Got an old litecoin mining farm? You're ready for VR..with last season's cards. Where did that bottle go?

"lamour42" wrote:
Something like just passing a reference to a texture in graphics memory that should be picked up by the SDK and simply rendered to the device.

That's basically what happens, but 'rendered to the device' currently means using the requisite rendering API, OpenGL or Direct3D.

"lamour42" wrote:
For many reasons this is not the case. In my opinion, what we have now is an SDK that interferes very much with your application, but doesn't tell us the details about it. That is not opaque, that is information hiding. Always a bad practice in software design.

This is a fundamental flaw with OpenGL (and probably with Direct3D as well, but I can't be certain). When you want to do something in OpenGL that requires specific state, you either have to aggressively set all the state values (which is undesirable as it can be needlessly expensive) or you need to ensure that you restore state to they way it was before you started (which is difficult, because getting the initial state means flushing the rendering pipeline and potentially introducing stalls). I believe this dichotomy is why there are so few libraries that back-end directly into a rendering API, as opposed to a framework that manages all the state.

"lamour42" wrote:
Compare this to client rendering.

Yes, in client rendering you're in direct control of the state at all times, so you don't have to face the issue.

"lamour42" wrote:
It may well be that SDK rendering will be the better choice at some day. Currently, I don't think it is mature enough already.

I suspect in the end it will come down to getting rendering extensions from the GPU manufacturers, which are already needed for other reasons (like prioritizing distortion rendering over other GPU tasks running concurrently). Client side distortion can't solve that problem.

"lamour42" wrote:

Debugging. Using Graphics debugging does not work with SDK mode for me. I get spurious exceptions in my own project when I try to start the graphics debugger. Even the World Demo does not allow for graphics debugging because I cannot capture a screen. (But I didn't see exceptions - obviously I am doing some things wrong in my project).

The oculus 'driver' is using the same hooks that the debugger uses. There is a conflict (who is hooking who?) so they step on each other, causing those crashes.

I was going to try a compiler switch, where all the oculus SDK stuff is not used, so there won't be hooks, and I can debug with pre-captured pose/HMD data.

I also noticed the 'dangling' state issues and the d3d debug runtime warnings/errors, all that error spam in the output is annoying when I'm trying to track down my own issues.

"jherico" wrote:
This is both a problem that is partially outside the control of the SDK and partly a symptom of their legacy code I think. Rendering APIs have issues with calls that fetch information out of the rendering engine because it typically depends on state that could be modified by rendering commands that have been issued, but not yet executed, so calling any kind of 'get state' call (at least on OpenGL) can cause synchronization points and generate stalls in the rendering pipeline.

I thought for most drivers, glGet functions generally stalled the CPU and processed all remaining commands in the pipeline, meaning that these synchronisation issues shouldn't be there (as in a sync may occur, but by time the value is retrieved from glGet the commands before have been processed)? I'm pretty sure (on the OpenGL side of things at least) that it's just bad state management in their SDK (I've only ever seen the exact same behaviour on different setups).