Set Stereo Separation to 0 when tracking type set to "Rotation Only"

I am using the OpenXR Plugin (1.8.2) with Unity 2022.3.4f1 and I need to disable position tracking. To do this, I set the tracking type in the TrackedPoseDriver component to "Rotation Only", but since there is still some stereo separation between the eyes, then rotating the head still causes some translation in the eye positions. The result is that a 3D character placed in "the center" of scene with a 360 video skybox appears to "float" in the scene.

How can set the eye/stereo separation to 0 so that when rotating the head, the eyes don't move?

I have tried setting Camera.stereoSeparation in Unity to 0, but this value seems to be ignored with the OpenXR Plugin. I've also tried setting the stereo view matrix on the camera to be the same for the left/right eyes, but this causes even more issues...

Any ideas?