cancel
Showing results for 
Search instead for 
Did you mean: 

Delta time variation jitter (caused by decoupled fixed time loop)

RektGamesPetterV
Explorer

Hi developing a game for Oculus Quest 2 and have some problems regarding jitter in movement that is caused by a variation of delta-time.

The jitter started when I introduced a fixed time loop of heavy operations. In my case I introduced a networking library that runs 60 times per second and therefore added some lag-spikes in the interval. But I have also added some artificial delays (sleep) to unitys-fixed update and notices the same behavior. My game still runs in the expected time-frame of 90 fps, but some frames take about 11 ms and others about 5 ms (example) which makes for a big variation in delta-time.

Variation in frame-times has not been a problem in traditional pc games I have worked with before. The delta time is either made almost constant by the use of v-sync. Or necessary to variate to compensate for the variation in time between each presentation of the frame. And movement results are smooth.

However it seems to be handled a bit differently on the Quest platform, I believe v-sync is not enabled, and instead Oculus use something called "OculusRuntime.WaitToBeginFrame" to keep a fixed frame-rate. The key difference is that "WaitToBeginFrame" is called in the beginning of the frame instead of how v-sync is called after the frame to do the actual waiting. I guess it is made this way to create as little input latency as possible before each rendered frame.

I also assume by looking at my profiler data, that the time to wait in "WaitToBeginFrame" is calculated based on the length of the previous frame. This would work great as long as frames are equally long but with variations in frame-time (as I have when introducing a decoupled network loop) this introduce even more variation by making a frame that already takes more time then the previous wait for the time saved by the previous.

So the problem I have now is that all movement that is moved in normal update by delta time is very choppy and jittery. It almost looks like the object is in two places at once because of the delta time variations. I am not sure how to fix this, since this normally wouldn't happen in a traditionell game.

Does anyone have any more insight or information that could help me?
Have I understand correctly how "WaitToBeginFrame" works? 

 

TL;DR: Has anyone here had problem with stutter on Oculus quest caused by delta time variation, which are present because of heavy operations in fixed time loops (fixed time lag-spikes). And knows any way to deal with it?

6 REPLIES 6

Not sure if it will help but totally randomly a friend forward this on to me just the other day. I've not tried to implement it yet, nor have I really had the kind of issues it supposedly solves, but it does sound similar to what you've written about.

using System.Diagnostics;
using UnityEngine;
using UnityEngine.XR;
using System.Collections.Generic;

public class CustomDeltaTime : MonoBehaviour
{
    private Stopwatch stopwatch;
    private float previousTime;
    private float refreshRate;
    private XRDisplaySubsystem displaySubsystem;

    void Start()
    {
        stopwatch = new Stopwatch();
        stopwatch.Start();
        previousTime = GetTimeInSeconds();

        // Get the XR Display Subsystem
        displaySubsystem = GetXRDisplaySubsystem();

        if (displaySubsystem != null)
        {
            // Initial refresh rate
            displaySubsystem.TryGetDisplayRefreshRate(out refreshRate);
        }
    }

    void Update()
    {
        float currentTime = GetTimeInSeconds();
        float deltaTime = currentTime - previousTime;
        previousTime = currentTime;

        // Use the delta time for controlling movement, animations, etc.
        // Example:
        transform.position += new Vector3(1, 0, 0) * deltaTime;

        // Refresh rate might change, so we update it each frame
        if (displaySubsystem != null)
        {
            displaySubsystem.TryGetDisplayRefreshRate(out refreshRate);
        }
    }

    private float GetTimeInSeconds()
    {
        // Adjust time scale according to refresh rate
        float timeScale = refreshRate != 0 ? 1.0f / refreshRate : 1.0f;
        return (float)stopwatch.ElapsedMilliseconds / 1000.0f * timeScale;
    }

    private XRDisplaySubsystem GetXRDisplaySubsystem()
    {
        List<XRDisplaySubsystem> displaySubsystems = new List<XRDisplaySubsystem>();
        SubsystemManager.GetInstances(displaySubsystems);

        if (displaySubsystems.Count > 0)
        {
            return displaySubsystems[0];
        }

        return null;
    }
}

 

Thanks for sharing 🙂

I am not sure this script in particular would help me or not, but working around Unitys built in time system and creating my own would probably work. But I would want to avoid it, since it could make integration with other third party systems problematic if they work with Unitys time system. It also feel kind of a hack.

Currently I kinda feel like "OculusRuntime.WaitToBeginFrame" forcing me out of Unitys ecosystem, preventing me to use the engine as expected. I had hope someone would point out a flaw I been doing so I could get back into the system, but right now maybe forcing myself around it is my only choice.

Netcode is a bit out of my league but one of the better description I've found is:

"WaitToBeginFrame, that means your app is doing something that causes a miss in the VSYNC deadline. In essence, the last frame gets reprojected and the Oculus runtime decides that you need to wait before starting the next frame. Otherwise you're going to start introducing latency in your app. That's all decided on by the Oculus runtime.

As to what causes missing that vsync deadline, that could be anything, from your app taking extra time on the CPU or GPU. One thing you can do is check the frame (or two) before the spike to see if there's anything suspect there."

The other thing that comes to mind is possibly trying to enable PhaseSync? https://developer.oculus.com/documentation/unity/enable-phase-sync

I'd recommend posting on the Unity forums, or possibly better the issue tracker and hope that a Unity engineer gets back to you. Sadly there are almost never any Meta people here on these forums. Good luck, and please do post again if you find a solution/explanation. 

Netcode is not really relevant to the problem, just wanted to have a clear purpose of why I do heavy operations at a fixed-rate outside of the update-rate.

I actually just read about PhaseSync, I hade somehow missed what it was. And in the documentation it sounds like it does exactly what I have had problem with. They actually even stated that

"If the app’s workload fluctuates violently or spikes frequently, Phase Sync may cause more stale frames than when Phase Sync is not enabled."

Which is exactly the problem I have been trying to report. Sadly I can not turn PhaseSync off, as it turned out it already was off... But it feels like it isn't, it feels like it is forced set to true somewhere as the explanation of the feature fits perfectly with what I am experiencing. 

Thanks for the recommendation, I will try to post in the Unity forum and hope I have better luck there.

RektGamesPetterV
Explorer

I run the logcat test sugested in the documentation (https://developer.oculus.com/documentation/unity/enable-phase-sync) to see if Phase Sync actually was on or not and discovered that it indeed was on. So the settings under PlayerSettings/XR Plug-in Management/oculus/android/Phase Sync seems to be ignored.

RektGamesPetterV
Explorer

I have have been able to find out that I'm able to turn off "Phase Sync", if I don't use the "XR Plugin" that is provided together with the "Oculus Intergration SDK" and also use Unity version 2021.2.6f1.

I am not sure at which Unity version the ability to toggle "Phase Sync" disappear again, but I can confirm that on unity version 2021.3.15f1, I am no longer able to turn "Phase Sync" off. Even when not using the "Oculus integration SDK" all together.

I have filed a bug report in Unity.