Handtracking fails after switching OVRCameraRigs

the handtracking works perfectly fine when open palms are facing the headset. but stops immediately when grabbing or rotating the hands. However the Oculus button still shows up and handtracking works normal when pausing the app.

I'm switching between OVRCameraRigs by instantiating two prefabs. both have handtracking and controller tracking enabled. The first one I've set up myself and the second one is from the vuplex webviev example scene for the quest.
After switching back from the vuplex-OVRCameraRig to mine and loading out the scene with vuplex components the tracking fails and switching back to controllers isn't detected either.

At this point I unload everything and make sure that all persistant objects are destroyed, before loading the first scene non-asynch. But the issue still persists. I've also removed the OVRManager from all OVRCameraRigs and added an OVRManager to an empty gameObject that is initiated in the first scene.