Skip to main content
kangsei
New Participant
July 23, 2025

How to implement tracking for real-world objects (6DoF) on Meta Quest 3 Passthrough?

  • July 23, 2025
  • 1 reply
  • 488 views

I'm working on a Unity project using Meta Quest 3 Passthrough, and I’d like to implement Vuforia-like functionality: recognizing real-world objects (like puzzle pieces) and tracking their full 6DoF pose (position + orientation) in real-time. I tried Unity Sentis but it only returns class labels and bounding boxes (e.g., ‘table’) and doesn't estimate 6DoF pose or detect if an object has been rotated.

Specifically:
- Should I train a custom 6D pose estimation model (e.g., YOLO) and run it with Sentis on-device?
- Are there any existing projects or SDK that support this?
- I'm currently working on an AR assistant that provides puzzle-solving guidance based on the user's behavior

Any advice, samples would be greatly appreciated.

    This topic has been closed for replies.

    1 reply

    mage2421148579
    mage2421148579
    Participant
    July 26, 2025

    I'm also trying to implement 6DoF tracking for real-world objects, but my tracking target is humans. I use MediaPipe to track human poses. I pass images through the camera API to MediaPipe to obtain the 0-1 coordinate system of human key points, and then use the EnvironmentRaycastManager to get the Z-axis values to achieve tracking of passers-by.

      Terms & ConditionsAccessibility statement