Delta time variation jitter (caused by decoupled fixed time loop)

Not sure if it will help but totally randomly a friend forward this on to me just the other day. I've not tried to implement it yet, nor have I really had the kind of issues it supposedly solves, but it does sound similar to what you've written about.

using System.Diagnostics;
using UnityEngine;
using UnityEngine.XR;
using System.Collections.Generic;

public class CustomDeltaTime : MonoBehaviour
{
    private Stopwatch stopwatch;
    private float previousTime;
    private float refreshRate;
    private XRDisplaySubsystem displaySubsystem;

    void Start()
    {
        stopwatch = new Stopwatch();
        stopwatch.Start();
        previousTime = GetTimeInSeconds();

        // Get the XR Display Subsystem
        displaySubsystem = GetXRDisplaySubsystem();

        if (displaySubsystem != null)
        {
            // Initial refresh rate
            displaySubsystem.TryGetDisplayRefreshRate(out refreshRate);
        }
    }

    void Update()
    {
        float currentTime = GetTimeInSeconds();
        float deltaTime = currentTime - previousTime;
        previousTime = currentTime;

        // Use the delta time for controlling movement, animations, etc.
        // Example:
        transform.position += new Vector3(1, 0, 0) * deltaTime;

        // Refresh rate might change, so we update it each frame
        if (displaySubsystem != null)
        {
            displaySubsystem.TryGetDisplayRefreshRate(out refreshRate);
        }
    }

    private float GetTimeInSeconds()
    {
        // Adjust time scale according to refresh rate
        float timeScale = refreshRate != 0 ? 1.0f / refreshRate : 1.0f;
        return (float)stopwatch.ElapsedMilliseconds / 1000.0f * timeScale;
    }

    private XRDisplaySubsystem GetXRDisplaySubsystem()
    {
        List<XRDisplaySubsystem> displaySubsystems = new List<XRDisplaySubsystem>();
        SubsystemManager.GetInstances(displaySubsystems);

        if (displaySubsystems.Count > 0)
        {
            return displaySubsystems[0];
        }

        return null;
    }
}

Netcode is a bit out of my league but one of the better description I've found is:

"WaitToBeginFrame, that means your app is doing something that causes a miss in the VSYNC deadline. In essence, the last frame gets reprojected and the Oculus runtime decides that you need to wait before starting the next frame. Otherwise you're going to start introducing latency in your app. That's all decided on by the Oculus runtime.

As to what causes missing that vsync deadline, that could be anything, from your app taking extra time on the CPU or GPU. One thing you can do is check the frame (or two) before the spike to see if there's anything suspect there."

The other thing that comes to mind is possibly trying to enable PhaseSync? https://developer.oculus.com/documentation/unity/enable-phase-sync

I'd recommend posting on the Unity forums, or possibly better the issue tracker and hope that a Unity engineer gets back to you. Sadly there are almost never any Meta people here on these forums. Good luck, and please do post again if you find a solution/explanation. 

I run the logcat test sugested in the documentation (https://developer.oculus.com/documentation/unity/enable-phase-sync) to see if Phase Sync actually was on or not and discovered that it indeed was on. So the settings under PlayerSettings/XR Plug-in Management/oculus/android/Phase Sync seems to be ignored.

I have have been able to find out that I'm able to turn off "Phase Sync", if I don't use the "XR Plugin" that is provided together with the "Oculus Intergration SDK" and also use Unity version 2021.2.6f1.

I am not sure at which Unity version the ability to toggle "Phase Sync" disappear again, but I can confirm that on unity version 2021.3.15f1, I am no longer able to turn "Phase Sync" off. Even when not using the "Oculus integration SDK" all together.

I have filed a bug report in Unity.