Delta time variation jitter (caused by decoupled fixed time loop)
Hi developing a game for Oculus Quest 2 and have some problems regarding jitter in movement that is caused by a variation of delta-time.
The jitter started when I introduced a fixed time loop of heavy operations. In my case I introduced a networking library that runs 60 times per second and therefore added some lag-spikes in the interval. But I have also added some artificial delays (sleep) to unitys-fixed update and notices the same behavior. My game still runs in the expected time-frame of 90 fps, but some frames take about 11 ms and others about 5 ms (example) which makes for a big variation in delta-time.
Variation in frame-times has not been a problem in traditional pc games I have worked with before. The delta time is either made almost constant by the use of v-sync. Or necessary to variate to compensate for the variation in time between each presentation of the frame. And movement results are smooth.
However it seems to be handled a bit differently on the Quest platform, I believe v-sync is not enabled, and instead Oculus use something called "OculusRuntime.WaitToBeginFrame" to keep a fixed frame-rate. The key difference is that "WaitToBeginFrame" is called in the beginning of the frame instead of how v-sync is called after the frame to do the actual waiting. I guess it is made this way to create as little input latency as possible before each rendered frame.
I also assume by looking at my profiler data, that the time to wait in "WaitToBeginFrame" is calculated based on the length of the previous frame. This would work great as long as frames are equally long but with variations in frame-time (as I have when introducing a decoupled network loop) this introduce even more variation by making a frame that already takes more time then the previous wait for the time saved by the previous.
So the problem I have now is that all movement that is moved in normal update by delta time is very choppy and jittery. It almost looks like the object is in two places at once because of the delta time variations. I am not sure how to fix this, since this normally wouldn't happen in a traditionell game.
Does anyone have any more insight or information that could help me?
Have I understand correctly how "WaitToBeginFrame" works?
TL;DR: Has anyone here had problem with stutter on Oculus quest caused by delta time variation, which are present because of heavy operations in fixed time loops (fixed time lag-spikes). And knows any way to deal with it?