Setting tracking origin to a previous state.

So I have a feature in the game where I what to temporarily change the tracking origin. When I activate this feature I can trivially do that by calling ovr_RecenterTrackingOrigin and that is fine. But when I exit this feature I want to go back to the original tracking origin. I don't simply want to reset again, as the HDM could be pointing any which way at this point.

I figured I could store the original tracking state, and then use ovr_SpecifyTrackingOrigin to go back. I can see from the documentation that "This transform is expected to be an offset to the most recent recentered origin", so it doesn't take an absolute origin. So I cannot simply use the origin from the original saved tracking state. Instead I think I need to calculate the delta that will take me there using the current origin and the saved one. However this doesn't seem to work either. i may just be getting lost in my quaternion math, but I figured I would ask here if this at least sounds like the right way to go about my original goal of getting back to the original tracking origin?

Regards

/Nis