Position and direction of Ray from hand offset?

I’m trying to have a gun type of thing attached to the players hand, and I have it somewhat working, but the origin of the ray and it’s direction seem to change based on the position and rotation of the hand. Sometimes it’s perfect, and sometimes it’s 3 feet to the right of my hand shooting 30 degrees to the right of where it should be, and anywhere in between. The rays origin is a cube that’s a child of RightHandAnchor and the direction is the transform.forward of right hand anchor. Does anyone know what’s happening here/how to fix it?

Edit: testing on Oculus Quest