Grabbing problems on Oculus Quest while using OVRPlayerController with LocalAvatarWithGrab

Hey there, we have a little problem while developing for the oculus quest. If we try to combine the OVRPlayerController with the LocalAvatarWithGrab prefab in Unity we get a strange problem. Grabbing works only sometimes. We need to grab an object very often to get it connected to the hand. When we use only the LocalAvatarWithGrab prefab (not combined with OVRPlayerController) everthing works like a charm. But we need a Character wich we can moove inside the game scene via controller. We put everything together with this hierachy:

OVRPlayerController
 -OVRCameraRig
    -TrackingSpace
      -LocalAvatarWithGrab
      -LeftEyeAnchor
      -CenterEyeAnchor
      -RightEyeAnchor
      -TrackerAnchor
      -LeftHandAnchor
      -RightHandAnchor

Grabbing works sometimes but not everytime. If i deactivate the OVRPlayerController script in the inspector grabbing objects works like expected. But with the script acitvated, and the ability to moove, grabbing is buggy somehow. Do you have any idea what we are doing wrong? We are using unity pro 2018.4.0f1 and the latest oculus integration (1.38) from the asset store. Best regards, Jan.