Headset position in real world for passthrough

I am trying to use the Quest Pro as an AR headset using the Passthrough. I want to be able to overlay virtual doubles of real-world objects such that I can then perform raycasting on them. I have made a calibration procedure that uses infrared cameras to find the transform from real world to unity world, but the calibration breaks between sessions. I wonder why?

How does the global virtual coordinates initialized with respect to the headset? Does the headset always start at 0,0,0 and the virtual world adjust to that? 

Also, I have physically marked my safety boundary on the floor and between headset restart notice that the boundary does not match my initial markings. Does that mean that between headset restarts the localization and mapping of my boundary is off and therefore my real world to virtual world calibration breaks?