Forum Discussion
How to implement tracking for real-world objects (6DoF) on Meta Quest 3 Passthrough?
I'm working on a Unity project using Meta Quest 3 Passthrough, and I’d like to implement Vuforia-like functionality: recognizing real-world objects (like puzzle pieces) and tracking their full 6DoF pose (position + orientation) in real-time. I tried Unity Sentis but it only returns class labels and bounding boxes (e.g., ‘table’) and doesn't estimate 6DoF pose or detect if an object has been rotated.
Specifically:
- Should I train a custom 6D pose estimation model (e.g., YOLO) and run it with Sentis on-device?
- Are there any existing projects or SDK that support this?
- I'm currently working on an AR assistant that provides puzzle-solving guidance based on the user's behavior
Any advice, samples would be greatly appreciated.
1 Reply
Replies have been turned off for this discussion
I'm also trying to implement 6DoF tracking for real-world objects, but my tracking target is humans. I use MediaPipe to track human poses. I pass images through the camera API to MediaPipe to obtain the 0-1 coordinate system of human key points, and then use the EnvironmentRaycastManager to get the Z-axis values to achieve tracking of passers-by.
