Practical Hand Tracking in Unity | Start Mentor Workshop
In this Meta Horizon Start workshop, Start Mentor Valem (Quentin) walks through practical hand tracking in Unity by building and comparing three core interaction models side-by-side: Finger Pinch (including pinch strength values and thresholds) Microgestures (tap + swipes, like an “invisible joystick”) Hand Pose Detection (custom shapes + hand orientation to avoid false matches) You’ll see how to set up a hand-tracking-ready Unity scene quickly using Building Blocks, then implement each input style with simple script patterns and live examples—so you can choose the best interaction method for your own XR app on Meta Horizon OS. This session was recorded in March 2026 as part of the Meta Horizon Start program. 🎬 CHAPTERS 👋 INTRODUCTION 🕒 00:00 - Welcome & workshop goals (comparing 3 hand tracking input models) 📋 PINCH (FINGER PINCH STRENGTH) 🕒 01:10 - What finger pinch is (and why index pinch is most reliable) 🕒 03:05 - Reading pinch strength (0–1) + using thresholds 🕒 05:10 - Interaction SDK examples (grab + ray select) 🕒 07:15 - Build demo: scene setup with Building Blocks + pinch-to-change-color spheres 🕒 10:50 - Smooth vs threshold pinch comparison + reliability notes (index vs other fingers) 📋 MICROGESTURES (INVISIBLE JOYSTICK) 🕒 14:10 - What microgestures are (tap + swipe directions) + use cases 🕒 16:00 - Reading microgesture type in code (switch/cases) 🕒 18:10 - Demo: show detected gesture in TextMeshPro 🕒 20:10 - Note: Interaction SDK locomotion uses microgestures (turn/teleport) 🕒 22:20 - Examples: character movement + UI navigation (and fixing missing EventSystem) 📋 HAND POSE DETECTION (SHAPE + ORIENTATION) 🕒 27:10 - Hand pose detection concepts: shape + orientation 🕒 28:40 - Create a custom Shape (thumbs up) using curl settings 🕒 31:10 - Trigger events when pose is recognized (active state → Unity events) 🕒 33:10 - Fixing thumbs up vs thumbs down: add wrist orientation (Transform Recognizer) 🕒 36:00 - Build thumbs down variant + wrap-up (choosing the right input model) 📚 RESOURCES ➡️ Meta Horizon Developer Forum: https://communityforums.atmeta.com/category/horizon-developer-forum ➡️ Developers Blog: https://developers.meta.com/resources/blog/ ➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/ 🔗 CONNECT WITH US Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter 💡 LEARN ABOUT THE META HORIZON START PROGRAM The Meta Horizon Start program provides intermediate and advanced developers with hands-on support and expert guidance to accelerate app development. Join a thriving community to access the tools and go-to-market resources you need to successfully deploy and grow your app on Meta Horizon OS. Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
How to Set Up Hand Tracking with the Meta All-in-One SDK | Mentor Workshop
If you're developing for Meta Quest and want reliable hand interactions, this is your starting point. In this session, Start Mentor Quentin Valembois (Valem) walks through the All-in-One SDK from foundational setup to advanced features, covering Building Blocks and Quick Actions before moving into gesture inputs and custom throwing physics. Plus, stick around until the end to catch the live Q&A featuring a member of Meta's Input Framework team. 🎬 CHAPTERS 🕒 00:00: Introduction 🕒 01:13: Setting Up Hand Tracking 🕒 08:41: The Interaction SDK 🕒 14:48: Automating Setup with Quick Actions 🕒 22:08: Triggering Inputs with Hands 🕒 26:29: Advanced Locomotion & Physics 🕒 33:22: Q&A with Meta's Input Framework Team 📚 RESOURCES 🔗 Featured GitHub Project: https://github.com/Meta-Horizon-Start-Program/MasterHandTracking 📺 Meta SDK V83: New Hand Tracking Locomotion Features: https://www.youtube.com/watch?v=V5BudZA9b9Q ➡️ Developers Blog: https://developers.meta.com/resources/blog/ ➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/ 🔗 CONNECT WITH US ➡️ Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter 💡 LEARN ABOUT THE META HORIZON START PROGRAM The Meta Horizon Start program provides intermediate and advanced developers with the resources, hands-on support, and expert guidance needed to accelerate their app development. Join a thriving community to get the tools and go-to-market guidance you need to successfully deploy and grow your app on Meta Horizon OS. Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
Get Started with 6 New AI Building Blocks in Meta SDK v83 | Mentor Workshop
Meta SDK v83 gives developers a faster path to production-ready AI and hand tracking features through drag-and-drop Building Blocks in Unity. In this Mentor Workshop, Start Mentor Quentin (Valem) walks through each new feature with live demos so you can see apply them right away. Every Building Block covered in this session is available now and ready to drop into your projects. 💡 By watching this video, you will learn: How to set up and connect the six new AI Building Blocks for speech-to-text, text-to-speech, LLM, and object detection How the new Fast Motion and Motion Upsampling optimizations automatically improve hand tracking accuracy for high-speed VR interactions How to implement controller-free locomotion using micro-gestures, walking sticks, climbing mechanics, and NavMesh teleportation How to use the Throw Physics Tuner and its preset profiles to stabilize thrown objects without custom physics code Recorded live in January 2026 as part of the Meta Horizon Start program. 🎬 CHAPTERS 👋 INTRODUCTION 🕒 00:00: Welcome & v83 Overview 🕒 01:42: AI Building Blocks Introduction 🤖 AI BUILDING BLOCKS IN UNITY 🕒 09:28: Implementing Text-to-Speech 🕒 11:26: Integrating Large Language Models 🕒 16:00: Multimodal AI with Image Input 🕒 18:50: Building a Conversational AI Manager 🔍 OBJECT DETECTION & RECOGNITION 🕒 23:33: On-Device vs. Cloud-Based Detection 🖐️ HAND TRACKING IMPROVEMENTS 🕒 27:14: Fast Motion & Latency Optimization 🕒 29:21: Locomotion with Micro-Gestures 🕒 33:24: NavMesh, Vignettes, and Comfort Settings 🕒 36:00: Walking Sticks Locomotion Mode 🕒 37:57: Climbing Mechanics & Architecture 🎯 THROW PHYSICS 🕒 43:10: Throw Tuner Profiles & Setup 🕒 46:18: Live Demo with Physics Behaviors 🧑💻 MEET YOUR MENTOR This session is led by Quentin (Valem), one of the most recognized voices in the Start community since joining the program in 2019. In that time, he’s helped thousands of developers master Unity and VFX through his Valem Tutorials YouTube channel, while launching games like Detective VR on the Meta Horizon Store through his own studio. 🌐 VALEM ON YOUTUBE ➡️ Valem Tutorials: https://www.youtube.com/@ValemTutorials 📚 RESOURCES ➡️ Developers Blog: https://developers.meta.com/resources/blog/ ➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/ 🔗 CONNECT WITH US ➡️ Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter 💡 LEARN ABOUT THE META HORIZON START PROGRAM The Meta Horizon Start program provides intermediate and advanced developers with the resources, hands-on support, and expert guidance needed to accelerate their app development. Join a thriving community to get the tools and go-to-market guidance you need to successfully deploy and grow your app on Meta Horizon OS. Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
How to Polish your XR Title : Onboarding, Optimization and Game Feel
Hi guys, I recently ran a workshop for the Meta Horizon Start Competition focused on polishing an XR title. At that stage of the hackathon, with only a week left before submission, one question mattered more than any other: How do you make the biggest impact in the least amount of time? This post is a written summary of that workshop aiming at answering this question around three topics: Player onboarding Optimization Game feel But why these three? Because when you are close to shipping, it becomes critical to look at your application through the eyes of a player, not a developer. For example : The build can behave very differently on device compared to the editor, which makes optimization and debugging essential. If players do not understand what to do, onboarding needs improvement. If it works but feels flat or unsatisfying, game feel needs polish. Together, these areas tend to give the highest return on investment when time is limited. Let’s start with player onboarding, using the Meta All in One SDK. Player Onboarding Player onboarding is how you teach players to understand and interact with your game. A useful principle to keep in mind is: The best tutorial is no tutorial. Ideally, players learn by doing. You introduce interactions one at a time, following the player’s natural progression through the experience. That said, refining this approach requires time and playtesting, which is often limited near the end of a project. When time is short, these faster onboarding tools can help communicate essential information clearly. Tooltips Short, spatialized text pointing at an object or interaction. Best used to highlight a short action. UI Panels The most basic solution, using Unity UI to display text, images, or short videos. Ghost Hands Very effective for hand tracking. You can visually demonstrate a hand pose or motion. Tip: With hand tracking, you can click on play, do a certain pose with your own hand, then drag the hand model from the hierarchy to the project window to save it as a prefab that you can reuse and that will directly have the pose you want to show. Voice Prompts Recording short voice lines can be faster than writing UI and often feels more natural. Keep them short and contextual. I provide a Unity package with a sample scene demonstrating each of these approaches here: https://drive.google.com/file/d/1n3IUzLMH6_60foStzgSdR5WignTkWUFh/view?usp=sharing Here are three more guidelines about onboarding : Optimization Optimization is crucial. It is not only about meeting Meta VRC requirements so your app can ship, but also about player comfort and enjoyment. Below is a simple optimization workflow that works well even for beginners and usually resolves most common issues. Step 1: Check your project settings Many performance problems come from incorrect project setup. Use the Project Setup Tool from the Meta SDK Ensure Single Pass Instanced is enabled Review URP settings and quality levels Bake your lights whenever possible For a full setup checklist, see this video: https://youtu.be/BeB9Cx_msKA?si=AjfnZdoPxH3jPxk- Project Validation Tool in Action Step 2: Check your scene complexity Triangle budgets vary depending on your content, but a good general target is around 150,000 triangles visible at once. Draw calls are just as important, and often more critical so keep them as low as possible (under 80). Ways to reduce complexity: Lower triangle count per model Use frustum and occlusion culling Add LODs to complex meshes Use batching techniques where possible For more information about the target fps, drawcalls and triangle count, go check out this meta documentation page: https://developers.meta.com/horizon/documentation/unity/unity-perf Step 3: Use OVR Metrics Install OVR Metrics and always test outside the Unity editor. OVR Metrics gives you real performance data on device, with live graphs and indicators. Here are the three most important data to track : FPS GPU and CPU usage Render Scale (must stay above 0.85 to meet VRC requirements) You can download it here : https://www.meta.com/en-gb/experiences/ovr-metrics-tool/2372625889463779/ Play your build and look for frame drops, reduced render scale, or sustained high GPU or CPU usage. Step 4: Find and remove your bottleneck The previous step tells you when and where performance drops occur. Now you need to understand why. Use the right tools: Unity Profiler Identify CPU bottlenecks, scripts taking too long, physics spikes, or garbage collection issues. Frame Debugger Analyze draw calls and rendering passes to understand what is actually being rendered each frame. Meta Runtime Optimizer Helps identify XR specific performance issues related to the runtime and rendering pipeline. Once you know the bottleneck, you can make targeted changes instead of guessing. Step 5: Last resort performance boosts When time is very limited, two features can significantly improve performance with minimal effort: Dynamic Resolution Fixed Foveated Rendering Both offer configurable levels to balance performance and visual quality. Be careful when lowering values too aggressively, visual quality can degrade quickly. Always remember that Render Scale must stay above 85 percent to pass VRC. Fixed Foveated Rendering Applied to Eye Texture Game Feel Game feel is often underestimated, but small improvements here can dramatically improve how polished your XR experience feels. Moreover having responsiveness also helps guide the player to understand your game and help onboarding. Haptic feedback Haptics add physical feedback to interactions and are extremely effective in XR. Triggering haptics is often just a single line of code. If you want more advanced effects, you can build layered patterns using a haptic tool or studio. Tutorial: https://youtu.be/RUUwWMkXFt0?si=1L92-NwIL9xy4CfJ Grab poses Hand grab poses let you enforce a custom hand pose when grabbing an object. The pose can vary depending on how or where the object is grabbed. For example, a cup grabbed by the body uses a different pose than grabbing it by the handle. This small detail greatly improves realism and comfort. Documentation: https://developers.meta.com/horizon/documentation/unity/unity-isdk-creating-handgrab-poses/ Sound design Sound is often forgotten, yet incredibly important. Add sounds to every interaction Use pitch variation for natural randomness Adjust volume based on the intensity of the action The Meta All in One SDK already includes UI focused audio, but Meta also provides a free audio pack with more general sounds: https://developers.meta.com/horizon/downloads/package/oculus-audio-pack-1/ Outro So here it is guys, these are the most impactful steps you can take to polish an XR title when time is limited. I hope this breakdown is helpful. Feel free to share your own tips and tricks below on how you polish your XR titles.Build Intuitive Experiences with Hands & Microgestures
You can build powerful immersive experiences with Hand Tracking using the Meta Interaction SDK. Watch as Meta Engineering Manager Jesse Keogh shows you the essential steps to go from a blank Unity project to a fully interactive prototype with UI and 3D object interactions. You’ll walk away with production-ready techniques for Quest-specific optimization and system-backed gestures like Pinch and Microgestures that deliver reliable input across all users. 💡After viewing this session, you’ll understand how to: Configure the Interaction SDK using Quick Actions Implement system-backed gestures like Pinch Streamline project creation with the Meta Quest Developer Hub Test interactions using the XR Simulator 🎬 CHAPTERS 👋 INTRODUCTION 🕒 00:00 - Introduction to the Hands Workshop 🕒 02:41 - Overview of Hand Tracking System Features ⚙️ CORE CONCEPTS 🕒 05:35 - System-Backed Gestures & Interaction SDK 🕒 08:25 - Tour of Interaction SDK Capabilities 🕒 11:48 - Design Principles & Best Practices 🛠️ Workflow & Demo 🕒 14:23 - Project Setup with Meta Quest Developer Hub 🕒 20:39 - Live Demo: Building UI Interactions 🕒 34:02 - Live Demo: Adding Grabbable 3D Objects ✅ FINAL THOUGHTS 🕒 37:15 - Q&A: Advanced Use Cases & Debugging 📖 HAND TRACKING EXAMPLES REFERENCED IN THIS VIDEO 🔖 Interaction SDK Overview: https://developers.meta.com/horizon/documentation/unity/unity-isdk-interaction-sdk-overview/ 🔖 Hand Tracking Design Guidelines: https://developers.meta.com/horizon/design/hands/ 📚 RESOURCES ➡️ Developers Blog: https://developers.meta.com/resources/blog/ ➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/ 🔗 CONNECT WITH US ➡️ Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter 💡 LEARN ABOUT THE META HORIZON START PROGRAM The Meta Horizon Start program provides intermediate and advanced developers with the resources, hands-on support, and expert guidance needed to accelerate their app development. Join a thriving community to get the tools and go-to-market guidance you need to successfully deploy and grow your app on Meta Horizon OS. Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
How to Understand your Player and their Space : Context Aware MR
Hey folks! I just ran last week a small workshop about Context-Aware MR on how to understand your player and their space. I wanted to drop a summary here, and I’m looking forward to your tips, ideas, or comments. 😊 But, here is the big question. What are the tool we can use to really “understand” a player? Here are the four musketeers of Context Aware MR : Player Input, MRUK, Depth API, PCA. 1) Player Input You might think player input is a bit trivial but there’s actually a lot we can extract from it to understand our player behaviour. Of course, we get the basics: the position and rotation of the head and controllers, plus input from buttons or triggers. But that’s just the beginning. With features like hand tracking and voice recognition, we unlock much more. Hand tracking lets us detect custom hand poses or gestures. Voice recognition allows for voice commands and even detecting voice loudness, which can be used to animate a character’s mouth or trigger actions like blowing out a candle. By combining head and controller tracking, we can figure out not only where the player is looking but also estimate their overall body pose. These are just a few examples, but the creative possibilities are huge. While these features also apply to standard VR, let’s now move to tools that are specific to Mixed Reality starting with our next musketeer : MRUK ! 2) MRUK To understand MRUK I need to first explain what the Scene Data is. The Scene Data is an approximation of the player's environment, set up outside of your app through the Meta system. It gives you access to either a full mesh or a simplified version of the room using labeled boxes that lets you identify if this elements is a wall, door, floors or a furniture. The Mixed Reality Utility Kit (MRUK) is a powerful set of tools built on top of Scene Data. It helps you place, align, and make virtual content interact with the real world. Here are some examples of what MRUK enables: Smart spawn points on specific surfaces (like the floor or walls) while avoiding obstacles (like furniture) Collision for your virtual content Navmesh to move object around the player’s space without bumping into real-world elements Destructible scene mesh effects Dynamic lighting effects on real world QR code and keyboard tracking And more... While MRUK is incredibly useful, keep in mind that Scene Data doesn’t update in real time. That’s where the Depth API comes in. 3) Depth API The Depth API gives you real-time depth maps of what the user is currently seeing. This allows you to occlude virtual objects behind real-world elements, making them feel like a natural part of the environment and greatly increasing immersion. It also comes with a Depth Raycast Manager, which lets you detect collisions at runtime with real objects perfect for dynamic content placement or interactions. It’s a great complement to the Scene Model, filling in the gaps that static scene data can’t cover. Despite its potential, it's still underused in many XR projects. 4) Passthrough Camera Access (PCA) We’ve had the first three tools for a while now. But recently, a game-changing feature was introduced: access to the passthrough camera! With access to the camera, you can: Read the live image as a texture to do color picking, light estimation, or apply visual effects like blur Feed the image to AI models for computer vision tasks like object detection It opens a direct bridge between the real world and AI and that's huge for MR development. Good news: starting with version v83, new building blocks are available to help you set up PCA easily in your project. To Conclude Player Input, MRUK, Depth API, and Passthrough Camera Access form a powerful toolbox for building context-aware MR experiences. And now, with tools like PCA, creativity is more accessible than ever. We can finally build apps and games that truly adapt to each user and their real space. Hope you enjoyed this little summary a nd that you learned something new along the way. Go check out the different link provided in the post if you want to learn more about our 4 musqueteers and if you have a tip on how you these features in your app share them down bellow! 😊 Usefuls link : https://developers.meta.com/horizon/documentation/unity/unity-isdk-interaction-sdk-overview https://developers.meta.com/horizon/documentation/unity/unity-mr-utility-kit-overview https://developers.meta.com/horizon/documentation/unity/unity-depthapi-overview https://developers.meta.com/horizon/documentation/unity/unity-pca-overview Have a nice day! 👋The Complete List of Sample Unity VR Projects
Hey guys, I wanted to put together a list of my favorite sample projects that you can grab and learn from. In my opinion, these projects are pure goldmines, they don’t just showcase design principles around specific features but also provide direct examples of how to use them, which is especially important right now for something like a hackathon. For an even larger collection of Meta samples, see the GitHub list of all Meta sample repos here: https://github.com/orgs/oculus-samples/repositories?type=all Let’s start with our first category, the interaction samples. Interaction Samples Meta XR All-In-One (Interaction SDK) Sample Links: https://github.com/oculus-samples/Unity-InteractionSDK-Samples https://assetstore.unity.com/packages/tools/integration/meta-xr-all-in-one-sdk-269657 Description: A comprehensive demo from Meta’s XR Interaction SDK featuring core VR interactions like poking, grabbing, raycasting, UI, and locomotion, all working together. Perfect to understand how to integrate both hands and controllers in one system. First Hand Link: https://github.com/oculus-samples/Unity-FirstHand Description: A full VR game demo focused on hand-tracked interactions. It showcases a complete Unity experience using the Interaction SDK with hand tracking as the main input and controller fallback. XR Interaction Toolkit Examples Link: https://github.com/Unity-Technologies/XR-Interaction-Toolkit-Examples Description: Unity’s official XR Interaction Toolkit samples showing how to implement locomotion, selection, grabbing, and UI interactions. A solid starting point for setting up XR Origin and interactor/interactable components. Move Fast Link: https://github.com/oculus-samples/Unity-MoveFast Description: A fast-paced VR fitness demo using hand tracking and the Interaction SDK. The sample shows how to build an energetic workout game with responsive, punch-based interactions. Whisperer Link: https://github.com/oculus-samples/voicesdk-samples-whisperer Description: A voice-controlled VR experience demonstrating the Meta Voice SDK. Use voice commands as part of gameplay to learn how to integrate real-time voice recognition into your own projects. Tilt Brush (Open Brush) Link: https://github.com/icosa-foundation/open-brush Description: An open-source continuation of Google’s Tilt Brush. Lets users paint and sculpt in 3D space — an excellent reference for creative VR tools and spatial drawing. Multiplayer & Social Samples VR Multiplayer Sample (Unity XRI) Link: https://docs.unity3d.com/Packages/com.unity.template.vr-multiplayer@2.0/manual/index.html Description: Unity’s official multiplayer VR template featuring a prebuilt scene, avatars, and networking setup using Netcode for GameObjects. Great for learning multi-user interactions in VR. Mixed Reality Multiplayer (XR Multiplayer) Sample Link: https://docs.unity3d.com/Packages/com.unity.template.mr-multiplayer@1.0/manual/index.html Description: A tabletop MR multiplayer demo that includes avatars, voice chat, and shared AR/VR spaces. Features games like balloon slingshot and chess while teaching MR networking and colocation concepts. Tiny Golf Link: https://github.com/Meta-Horizon-Start-Program/Tiny-Golf Description: A free-to-play multiplayer mini-golf VR game created for the Meta Start program. Demonstrates basic physics, scoring, and networked multiplayer. Ultimate Glove Ball Link: https://github.com/oculus-samples/Unity-UltimateGloveBall Description: A VR e-sport showcase demonstrating multiplayer, avatars, voice, and in-app purchases. Integrates Photon networking and Oculus social APIs, making it a great reference for social competitive games. Spirit Sling Link: https://github.com/oculus-samples/Unity-SpiritSling Description: A social MR tabletop game letting players place a shared game board in real space and invite friends to join. Highlights Avatars SDK and MR colocated play. Decommissioned Link: https://github.com/oculus-samples/Unity-Decommissioned Description: A social-deduction VR game inspired by titles like Among Us. Shows how to handle multiplayer lobbies, Oculus invites, and social APIs in a networked Unity project. Mixed Reality (MR) Samples A World Beyond (Presence Platform Demo) Link: https://github.com/oculus-samples/Unity-TheWorldBeyond Description: A full MR showcase combining Scene Understanding, Passthrough, hand tracking, voice input, and spatial audio. A must-see for developers building immersive MR scenes blending real and virtual spaces. Phanto (MR Reference App) Links: https://github.com/oculus-samples/Unity-Phanto https://developers.meta.com/horizon/blog/phanto-unreal-showcase/ Description: An MR reference app focused on environmental awareness. Uses the Scene Mesh and MR APIs to blend gameplay with real-world geometry. Unity Discover (featuring Drone Rage and others) Links: https://www.meta.com/en-gb/experiences/discover/7041851792509764/ https://github.com/oculus-samples/Unity-Discover Description: A collection of MR showcase mini-experiences like Drone Rage. Demonstrates MR features including Passthrough, Spatial Anchors, and Shared Anchors in various game prototypes. MR Motifs Link: https://github.com/oculus-samples/Unity-MRMotifs Description: A library of MR “motifs”, small, reusable templates showcasing mechanics such as passthrough transitions, colocated multiplayer, and instant content placement. Cryptic Cabinet Link: https://github.com/oculus-samples/Unity-CrypticCabinet Description: A short MR escape-room experience that adapts to your room’s layout. Demonstrates interactive storytelling in mixed reality using environmental awareness. Passthrough Camera API Samples Link: https://github.com/oculus-samples/Unity-PassthroughCameraApiSamples Description: A sample project demonstrating how to access and process Quest’s Passthrough camera feed for effects, object detection, and image manipulation. Tool and Utilities Asset Streaming Link: https://github.com/oculus-samples/Unity-AssetStreaming Description: An open-world streaming sample that shows how to dynamically load content using Addressables and LOD systems — ideal for maintaining performance in large VR environments. Shader Prewarmer Link: https://github.com/oculus-samples/Unity-ShaderPrewarmer Description: A utility sample that preloads shader variants at startup to eliminate hitching or stutters when shaders first compile — an important optimization for smooth VR performance. Complete Game Showcase Northstar Link: https://github.com/oculus-samples/Unity-NorthStar Description: A complete VR game showcasing advanced interaction and visual technique for VR. Featuring rope physics, narrative story telling, lip sync and more.VR Hand Interactions Explained: Developer Deep Dive for Meta Quest (Unity & Unreal)
Unlock the full potential of hand interactions on Meta Quest in this technical deep dive from members of the Meta Interaction SDK team. Product Designer Stella Mühlhaus and Software Engineer Dave Nelson explore the core principles, best practices, and powerful tools available to create intuitive and engaging hands-first interactions. 💡 In this deep dive, you’ll learn: The key advantages of hands over controllers: embodiment, ease of use, and flexibility. How to implement core mechanics like Direct Touch, Grabbing, Hand Posing, and Gestures. Advanced techniques for Locomotion, Improved Throwing, and physics-based Natural Interactions. The latest system-level improvements in hand tracking, including Hands 2.3 and fast motion updates. A complete overview of the Interaction SDK for both Unity and Unreal Engine. Whether you're starting a new project or adding hand tracking to an existing app, this session provides the expert guidance you need to make your hands interactions feel amazing. 🎙️ SPEAKERS Stella Mühlhaus, Product Designer Dave Nelson, Software Engineer ▶️ VIDEO CHAPTERS 👋 Introduction 🕒 00:00 - Welcome & Speaker Introductions 🕒 00:27 - Agenda Overview 🕒 00:47 - Why Build for Hands? Industry Trends & Core Benefits ✨ Core Interaction Mechanics 🕒 05:16 - Direct Touch & UI Interaction 🕒 06:04 - Grabbing, Posing & Object Interaction 🕒 07:27 - Pose & Gesture Recognition 🕒 08:14 - Hand Menus, Microgestures & Multimodal Input 🚀 Upcoming Features & Experiments 🕒 10:40 - Improved Throwing Mechanics 🕒 11:59 - Locomotion Experiments (Telepath, Walking Sticks, Climbing) 🕒 13:31 - Natural Interactions: Physics-Based Hands 🕒 15:39 - CARL: Custom Action Recognition Library ⚙️ System-Level Hand Tracking Improvements 🕒 17:21 - Overview of Core Tracking Updates 🕒 17:49 - Microgesture Improvements & Hands 2.3 🕒 19:55 - Fast Motion Improvements for High-Speed Tracking 🛠️ Building for Hands: Best Practices & Tools 🕒 21:25 - Best Practices for Designing Hands-First Experiences 🕒 25:31 - Deep Dive: Interaction SDK for Unity & Unreal 🕒 28:27 - Resources: Sample Apps to Get You Started 🕒 29:40 - Closing Remarks 📚 RESOURCES ➡️ Hands Design Guidelines: https://developers.meta.com/horizon/design/hands/ ➡️ Interaction SDK: https://developers.meta.com/horizon/documentation/unity/unity-isdk-interaction-sdk-overview ➡️ Microgestures Open XR extension: https://developers.meta.com/horizon/documentation/unity/unity-microgestures 🔗 CONNECT WITH US ➡️ Developers Blog: https://developers.meta.com/resources/blog/ ➡️ Meta Quest Developer Hub: https://developers.meta.com/horizon/documentation/unity/ts-mqdh/ ➡️ Sign up to get the latest news from Meta Horizon: https://developers.meta.com/horizon/newsletter 💡 LEARN ABOUT THE META HORIZON START PROGRAM The Meta Horizon Start program provides intermediate and advanced developers with the resources, hands-on support, and expert guidance needed to accelerate their app development. Join a thriving community to get the tools and go-to-market guidance you need to successfully deploy and grow your app on Meta Horizon OS. Apply to Start today: https://developers.meta.com/horizon/discover/programs/start
