Skip to main content
JCorfer
Inspiring
June 25, 2026
Solved

Meta Quest 3 Multimodal: How to prevent Controller from momentarily following Hand movement? (Unity)

  • June 25, 2026
  • 2 replies
  • 132 views

Hi everyone,

I am currently developing a Proof of Concept (PoC) in Unity for Meta Quest 3 where I need to use both hands and both controllers simultaneously and completely independently.

I have successfully configured the project to render both the hands and the controllers at the same time, correctly positioned in space. However, I am running into a specific visual/tracking issue:

When I move my physical hand away from a controller, the virtual controller momentarily tries to follow/snap to the hand's position before lagging behind and eventually snapping back to its actual, real-world physical position.

  • My Question: Is this intended multimodal fusion behavior (algorithmic blending) or a known tracking bug?
  • My Goal: I need the tracking positions of the hands and the controllers to be completely independent of each other at all times, without this "rubber-banding" or tethering effect when they separate.

Environment Setup:

  • SDK Version: Unity Meta All-In-One SDK 203.0.0
  • Engine: Unity
  • Feature Enabled: Simultaneous Hands and Controllers (Multimodal)

Has anyone managed to completely decouple the controller tracking from the hand tracking under the new multimodal system, or is this filtering hardcoded into the runtime? Any advice on how to bypass this lag/blending via SDK settings or script would be greatly appreciated.

Thanks in advance!

    Best answer by yael3D

    Hello! I needed a similar feature for a project and ran into the exact same issue.
    SDK v85

    Probably hacky, but what worked for me was navigating to: [BuildingBlock] OVRInteractionComprehensive -> RightInteractions -> SyntheticControllerData
    Inside the Synthetic Controller In Hand component, you can do one of these three things to stop the blending:

    • Set the Apply Modifier field to false
    • Remove the Optionals / Synthetic Hand reference
    • Disable the Active State Unity Event Wrapper component

    To retrieve the position of the controller, I use: [BuildingBlock] OVRInteractionComprehensive -> OVRRightControllerVisual. I also disabled its Active State Unity Event Wrapper component, as otherwise I would lose the controller's position when grabbing or pinching an object.

    Side note on my own Multimodal experience: My biggest issue right now when having both active is that interactions/grabbables work perfectly with hand tracking, but when using the controller to grab/pinch, it uses the position of the IRL hand rather than the controller. This causes the grab parent position to be jittery and unreliable. Still looking for a fix for that!
    (Used AI for reformatting my answer)

    2 replies

    yael3DAnswer
    New Participant
    June 26, 2026

    Hello! I needed a similar feature for a project and ran into the exact same issue.
    SDK v85

    Probably hacky, but what worked for me was navigating to: [BuildingBlock] OVRInteractionComprehensive -> RightInteractions -> SyntheticControllerData
    Inside the Synthetic Controller In Hand component, you can do one of these three things to stop the blending:

    • Set the Apply Modifier field to false
    • Remove the Optionals / Synthetic Hand reference
    • Disable the Active State Unity Event Wrapper component

    To retrieve the position of the controller, I use: [BuildingBlock] OVRInteractionComprehensive -> OVRRightControllerVisual. I also disabled its Active State Unity Event Wrapper component, as otherwise I would lose the controller's position when grabbing or pinching an object.

    Side note on my own Multimodal experience: My biggest issue right now when having both active is that interactions/grabbables work perfectly with hand tracking, but when using the controller to grab/pinch, it uses the position of the IRL hand rather than the controller. This causes the grab parent position to be jittery and unreliable. Still looking for a fix for that!
    (Used AI for reformatting my answer)

      JCorferAuthor
      Inspiring
      June 26, 2026

      Hi yeal3D!! Removing the Optionals / Synthetic Hand References have been enough to resolve the issue!

      Thanks so much for your help!

      Terms & ConditionsAccessibility statement