Passthrough Camera Access (PCA) + AI Building Blocks on Quest 3 | Developer Workshop
With Passthrough Camera Access (PCA), your Quest 3 app reads the real-world camera feed as GPU textures you can use in Unity for mixed reality. In this session, the Meta XR team walks through the new PCA component in the Meta XR SDK and its privacy-first design that replaces the internal Medici prototype. You’ll see how PCA connects with AI Building Blocks for object detection and guidance, and how to use these tools in your submission to the 2025 Meta Horizon Start Developer Competition.
💡 By viewing this session, you’ll learn how to:
- Set up the PassthroughCameraAccess component in Unity and choose the right permissions and resolution for your use case.
- Map 2D coordinates from the camera to 3D rays with depth so virtual content stays locked to real surfaces.
- Use AI Building Blocks for on-device object detection and LLM prompts that respond to what the camera sees.
- Plan a PCA-powered prototype that meets the requirements of the Meta Horizon Start Developer Competition.
🎬 CHAPTERS
👋 INTRODUCTION & OVERVIEW
🕒 00:00 - Introduction and Agenda
🕒 00:52- What Is Passthrough Camera Access (PCA)?
🕒 03:20- Inspiring Use Cases: Industry & Community Projects
🛠️ TECHNICAL IMPLEMENTATION
🕒 07:29 - Technical Deep Dive: The New PCA Component in SDK v81
🕒 12:10 - New & Upcoming PCA Features
🕒 15:52 - Mastering Coordinate Systems: 2D to 3D and 3D to 2D
🤖 AI INTEGRATION & OPTIMIZATION
🕒 19:47 - Introducing the AI Building Blocks
🕒 22:51 - Deep Dive: The Object Detection Building Block
✅ Q&A AND BEST PRACTICES
🕒 24:46 - Competition Details & Q&A Kick-off
🕒 28:14 - Q&A: Handling Latency and Inspiring Use Cases
🕒 40:44 - Q&A: PCA vs. Scene API & Visual Design Best Practices
🕒 45:46 - Q&A: Offline Voice Recognition & Final Remarks
📚 RESOURCES
➡️ Developers Blog: https://developers.meta.com/resources/blog/
➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/
🔗 CONNECT WITH US
➡️ Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter
💡 LEARN ABOUT THE META HORIZON START PROGRAM
The Meta Horizon Start program provides intermediate and advanced developers with the resources, hands-on support, and expert guidance needed to accelerate their app development. Join a thriving community to get the tools and go-to-market guidance you need to successfully deploy and grow your app on Meta Horizon OS.
Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
