Skip to main content
jimwe
Up and Coming
January 5, 2016

ovr_SubmitFrame execution time

Hi,

I am trying to improve the performance of an application that is soon to be released. My problem is that every 5-20th frame ovr_SubmitFrame takes a "long time" to execute. 

    
double st = ovr_GetTimeInSeconds();
ovrResult result = ovr_SubmitFrame( od->hmd, 0, &viewScaleDesc, &layers, 1 );
double dt = ovr_GetTimeInSeconds() - st;
if (dt < 0.01) {
    printf("Submit oculus frame OK dt: %f\n", dt);
} else {
    printf("Submit oculus frame ****TOO SLOW**** dt: %f\n", dt);
}


Here is a trace from a sequence of frames:

Submit oculus frame ****TOO SLOW**** dt: 0.018046
Submit oculus frame OK dt: 0.008834
Submit oculus frame OK dt: 0.009257
Submit oculus frame OK dt: 0.009368
Submit oculus frame OK dt: 0.009138
Submit oculus frame OK dt: 0.008067
Submit oculus frame OK dt: 0.007674
Submit oculus frame OK dt: 0.007736
Submit oculus frame OK dt: 0.006971
Submit oculus frame OK dt: 0.006168
Submit oculus frame OK dt: 0.004738
Submit oculus frame OK dt: 0.004581
Submit oculus frame OK dt: 0.004082
Submit oculus frame OK dt: 0.003918
Submit oculus frame OK dt: 0.004033
Submit oculus frame OK dt: 0.003598
Submit oculus frame ****TOO SLOW**** dt: 0.015026
Submit oculus frame OK dt: 0.009811
Submit oculus frame OK dt: 0.009004
Submit oculus frame OK dt: 0.008599
Submit oculus frame OK dt: 0.008140
Submit oculus frame OK dt: 0.007428
Submit oculus frame OK dt: 0.008093
Submit oculus frame OK dt: 0.007579
Submit oculus frame OK dt: 0.007019
Submit oculus frame OK dt: 0.006299
Submit oculus frame OK dt: 0.006645
Submit oculus frame OK dt: 0.006473
Submit oculus frame OK dt: 0.006101
Submit oculus frame OK dt: 0.005045
Submit oculus frame OK dt: 0.003781
Submit oculus frame OK dt: 0.004333
Submit oculus frame OK dt: 0.002795
Submit oculus frame ****TOO SLOW**** dt: 0.014661
Submit oculus frame OK dt: 0.008805
Submit oculus frame OK dt: 0.008262
Submit oculus frame OK dt: 0.007425
Submit oculus frame OK dt: 0.006468
Submit oculus frame OK dt: 0.006098
Submit oculus frame OK dt: 0.005475
Submit oculus frame OK dt: 0.004870
Submit oculus frame ****TOO SLOW**** dt: 0.018123

Here is a short video showing the problem (the VR visual experience is of course much than shown in the video):

http://screencast.com/t/i2brpuZwnu

My question is: Does anybody have a good explanation to why this happens? Is there anyway to avoid it?

I am experiencing stuttering every now and then and it seems to be related to these "slow" ovr_SubmitFrame calls.

System info:

- Windows 8 64-bit
- Nvidia 770 GTX (driver: NVIDIA Forceware 358.91)
- Oculus SDK 0.8
- DK2

Thanks,
/J

    8 replies

    thewhiteambit
    Inspiring
    January 5, 2016
    Looks like you are not in time for V-Sync, so it has to wait for the next V-Sync (+13ms)
    jimweAuthor
    Up and Coming
    January 5, 2016
    Thanks! Would you please point me in a direction where I can read more about how Oculus deals with V-sync?

    According to the doc ovr_SubmitFrame(..) works like this:

    "ovr_SubmitFrame triggers distortion and processing which might happen asynchronously. The function will return when there is room in the submission queue and surfaces are available. Distortion might or might not have completed."

    So it seems to me like it might take more time because there is no room left in the submission queue. This might sound strange but does this mean we are feeding the Oculus with frames a little bit too fast..? Maybe I should add a delay and feed the Oculus with new frames less often.

    More input would be appreciated!
    Thanks,
    J
    thewhiteambit
    Inspiring
    January 5, 2016
    Yes, this is what I thougt. Your render to fast, and overdrive the submitFrame resulting in throttle with false latency -> stutter.

    I would prefer, If you could at least overdrive SubmitFrame, so the last frame will be replaced with a newer one and less TimeWarp. Maybe good handling of frameIndex (not on the buffers, the one for measuring TimeWarp) could at least remove the feel of stutter. Also output the result of ovr_SubmitFrame could help. In best case it is something different just before the long frame.
    thewhiteambit
    Inspiring
    January 7, 2016
    We have a sawtooth visible in the "App-Trk to Mid-Photon" because of some strange behavior. I Guess we are rendering a bit slower than the SubmitFrame function takes new ones:

    Latency Jump.png

    So you can see, how render latency slowly goes down from 26ms to 14ms - but instead of keeping latency that low, it periodically jumps up. This makes sense somehow, since after rendering slower than the 75Hz, the frame calculated late must be calculated with more latency to match the frame time it will be presented.

    But I guess, these problems are somehow related... we can also render faster, but this (even rendering slightly faster) makes the rendering stall with high latency frames. I would love to render faster and give the runtime a new picture if the old one was not used, having low latency without lost frames.

    I will go after this oddity once I have time and keep you updated, I guess this is related somehow..

    PS: You could also watch the timing with QueueAhead disabled. This may lets you see more detailed timing info on the fast frames...
    jimweAuthor
    Up and Coming
    January 8, 2016
    Thanks for all the input! I will look closer at the "Performance Head-Up Display" (https://developer.oculus.com/documentat ... ts/dg-hud/). This section also added some information related to my previous question concerning V-sync and the Rift:

    "Because the compositor is locked to vsync, it will never exceed the native HMD refresh rate. But, if the compositor fails to finish a frame on time, it can drop below the HMD refresh rate."

    If anyone is looking for how to use the "Performance Head-Up Display" here is some sample code: 

    ovrPerfHudMode PerfHudMode = ovrPerfHud_LatencyTiming;
    ovr_SetInt(Hmd, OVR_PERF_HUD_MODE, (int)PerfHudMode);


    where ovrPerfHudMode_ can be: 

    typedef enum ovrPerfHudMode_
    {
        ovrPerfHud_Off                = 0,  
        ovrPerfHud_LatencyTiming      = 1,  
        ovrPerfHud_RenderTiming       = 2,  
        ovrPerfHud_PerfHeadroom       = 3,  
        ovrPerfHud_VersionInfo        = 4,  
        ovrPerfHud_Count,                   
        ovrPerfHud_EnumSize = 0x7fffffff    
     } ovrPerfHudMode;


    I will update this thread if I come across anything I think might be of interest to someone else.

    Thanks,
    /J
    jimweAuthor
    Up and Coming
    January 8, 2016
    Ok, by using the "Performance Head-Up Display" I can see that the stuttering definitely comes from the compositor missing V-sync. No V-sync miss - no stuttering.

    I think it is time for some details about our rendering approach. We are rendering using OpenGL and our rendering approach basically looks like this:

    1. Start a new Oculus frame (update camera positions by retrieving HMD tracking data).

    2. Render to two offscreen frame buffers (one for each eye)
    3. Call openGLFinish(..) to make sure all rendering commands have been executed. We want to make sure the eye textures have been updated.
    4. Copy the rendered images to the oculus eye render textures using glCopyTexSubImage2D(..).
    5. Submit the frame.

    6. End oculus frame.

    Now, openGLFinish(..) may (obviously) take some time to execute. I have three questions concerning this approach:

    1. Is calling openGLFinish(..) a good idea? To me it makes a lot of sense calling it because this way we can make sure we feed the oculus with fresh offscreen frame buffer contents.

    2. How long may "App CPU Render Time" be before it becomes a problem?

    3. Let say my "App CPU Render Time" always is below 10 ms (with openGLFinish(..) being used). Should that not be enough to make it before the Oculus limit of 13ms for each frame is reached? I guess it depends on when we start to render the frame in relation to the 75Hz interval of the Oculus, so here is the actual question: Is there anyway to enforce to keep the "App rendering" in sync with the Oculus 75 Hz? I mean so the app starts rendering in the beginning of the Oculus 75Hz interval making sure that 10ms for "App rendering" (leaving 3ms for Oculus) would always make it within the 75 Hz interval.

    Please ask away if you feel I am being unclear in my questions.

    Thanks,
    /J
    thewhiteambit
    Inspiring
    January 8, 2016
    "jimwe" wrote:

    Now, openGLFinish(..) may (obviously) take some time to execute. I have three questions concerning this approach:

    1. Is calling openGLFinish(..) a good idea? To me it makes a lot of sense calling it because this way we can make sure we feed the oculus with fresh offscreen frame buffer contents.

    Normally you don't have to under most conditions the driver will know. Just try it without :)
    http://stackoverflow.com/questions/2143 ... s-glfinish

    2. How long may "App CPU Render Time" be before it becomes a problem?

    At 75Hz you have about 13ms for everything including oculus distortion rendering (1-2ms)


    3. Let say my "App CPU Render Time" always is below 10 ms (with openGLFinish(..) being used). Should that not be enough to make it before the Oculus limit of 13ms for each frame is reached? I guess it depends on when we start to render the frame in relation to the 75Hz interval of the Oculus, so here is the actual question: Is there anyway to enforce to keep the "App rendering" in sync with the Oculus 75 Hz? I mean so the app starts rendering in the beginning of the Oculus 75Hz interval making sure that 10ms for "App rendering" (leaving 3ms for Oculus) would always make it within the 75 Hz interval.
    /J


    With QueueAhead you even can start slightly in advance. But if there are prolonged draw calls to the GPU (AMD has better preemtion at this level now than nVidia), the rendering can not happen in time sometimes. I would take away the Flush and try different attempts to copy the textures (Framebuffer-Draw-Blit or Fullscreen-Quad-Shader) to see if this helps. You can also decouple rendering the framebuffer from transfereing to the oculus runtime and have some async timewarp whenever you miss a frame.
    jimweAuthor
    Up and Coming
    January 14, 2016
    Thanks for the input.

    For us, openGLFinish(..) has made the rift experience much more pleasant. The only downside is when it starts to take too much time to do the actual rendering (missing V-sync).

    I will investigate wheter the refresh rate of the monitor can affect the number of V-sync misses we see. Currently I am using 60Hz monitors. Maybe 120 Hz will make a difference. We will see.

    Thanks,
    /J
    Terms & ConditionsAccessibility statement