Forum Discussion
SDK-side distortion, OpenGL & glFinish()/glFlush()
I've been brushing up on my GL skills, and I was wondering about the flow through the GL distortion pipeline in the SDK. I'm fundamentally unhappy with the idea of glFlush()/glFinish() being used for synchronization of any kind and even more so with the busy-wait mechanisms designed to block the client from beginning a new frame until the SDK decides the time is right.
It seems like one of the fundamental problems you're trying to solve is to ensure that when the next frame starts you have a decent idea of when it's going to be rendered, and thus can provide the most accurate head pose information. Right now this is done by flushing and busy-waiting in endFrame(), but this demonstrably causes dropped frames and a lower overall framerate on my desktop machine. I'm thinking there's a better way.
In the distortion rendering, you could instantiate a fence sync object after doing the draw, or performing the buffer swap, and use a queue of unsignaled sync objects as a count of pending frames, using that to determine prediction amount required for future rendering calls. You could expose the number of pending frames to the client, giving them the opportunity to perform other work if there were more than 1, meaning that they're too far ahead of the rendering pipeline.
Just a thought.
It seems like one of the fundamental problems you're trying to solve is to ensure that when the next frame starts you have a decent idea of when it's going to be rendered, and thus can provide the most accurate head pose information. Right now this is done by flushing and busy-waiting in endFrame(), but this demonstrably causes dropped frames and a lower overall framerate on my desktop machine. I'm thinking there's a better way.
In the distortion rendering, you could instantiate a fence sync object after doing the draw, or performing the buffer swap, and use a queue of unsignaled sync objects as a count of pending frames, using that to determine prediction amount required for future rendering calls. You could expose the number of pending frames to the client, giving them the opportunity to perform other work if there were more than 1, meaning that they're too far ahead of the rendering pipeline.
Just a thought.
1 Reply
- Thanks, jherico. Sync objects would indeed tell us when commands retired with fewer pipeline stalls. Unfortunately, without glWaitSync(..), they would make it more difficult to schedule distortion rendering right before the vblank.
"jherico" wrote:
It seems like one of the fundamental problems you're trying to solve is to ensure that when the next frame starts you have a decent idea of when it's going to be rendered, and thus can provide the most accurate head pose information.
The only reason we flush and wait is that it lets us sample the actual orientation and adjust the projection immediately before scan-out. It's true that this adds a pipeline bubble of a couple of milliseconds between your rendering and our distortion. But it reduces head motion-to-photons latency, which your brain is very sensitive to. Currently, there's too much error in poses that were predicted even just one frame ahead. We try to make the rendered image reflect actual data that is sensed just a millisecond or two before the display begins to show it.
The only reason you can't start rendering the next frame until after we perform distortion is that we are sharing a single context. You may recall that we're working toward an asynchronous timewarp technique, which performs distortion from a separate thread and context. Once that is available, you'll be able to call ovrHmd_EndFrame(..) and begin rendering the next frame without the stalls that you see today. Sync objects may be helpful there.
That being said, how much GPU time is the flush currently burning for your app? Once you have rendered the left and right eye textures, you could potentially get started on the next frame, as long as you call ovrHmd_EndFrame(..) several milliseconds before the next retrace. That's in the ballpark of 9ms after the previous call returned. You couldn't make extra eye begin/end calls, but you could set up state and even do some rendering with a camera pose of your own sampling.
