AI Algorithm Inquiry in VR Soccer Simulation

We are Abocado, a Meta Horizon Start member developing RoboTwin XR, an AI-powered VR robot simulation and smart farm education platform, for the Meta VR Start Developer Competition 2026, and Meta VR Glasses, using WebXR, Three.js, and Meta's Immersive Web SDK (IWSDK).

We are preparing to submit RoboTwin XR to the Productivity category of the Meta VR Start Developer Competition 2026 and would appreciate clarification regarding the hand-tracking requirements.

1. Nature of Our Application

1. RoboTwin XR is an immersive VR platform designed for AI robotics education, simulation, and performance evaluation through interactive robot soccer games.

2. It enables users to develop and test reinforcement learning, multi-agent AI cooperation, and autonomous decision-making algorithms by multilanguage.

3. The platform supports robot locomotion, motion control optimization, obstacle avoidance, and real-time strategy evaluation in virtual environments.

4. By integrating digital twins, AI voice interaction, and VR hand tracking, RoboTwin XR provides an interactive learning environment connecting virtual simulations with real-world robotics.

5. Its long-term vision is to deploy trained AI robotic agents in urban smart farms for remote cultivation, autonomous monitoring, crop management, and intelligent agricultural automation.

Unlike conventional VR games, RoboTwin XR primarily leverages multilingual AI-powered voice recognition, conversational control, and real-time commentary by an AI announcer to explain AI algorithms, reinforcement learning strategies, and multi-robot cooperation, providing an immersive and interactive educational experience.

For example, users can ask questions in their preferred language, such as:

•   What algorithms are currently being used in this soccer simulation? 

An AI-powered virtual avatar understands these questions and responds in the user's language, presenting relevant real-time information within the VR environment.

2. Hand-Tracking Implementation

We understand that the competition emphasizes hand-tracking interaction.

We are actively implementing hand-based controls wherever practical, including:

• Selecting control units and 3D objects.

• Navigating menus and information panels.

• Interacting with digital twin visualizations.

• Accessing game monitoring functions.

However, certain advanced functions, particularly multilingual AI conversations, complex game algorithm queries, and collaborative discussions, are more naturally and efficiently performed through voice interaction.

3. Request for Clarification

We would appreciate your guidance on the following questions:

1. Can an application qualify for the Productivity category if multilingual AI voice interaction is its primary interface, while hand tracking is provided for applicable navigation and interaction functions?

2. Does the hand-tracking requirement mean that every function must be fully executable using hand gestures alone, or can voice interaction be used for specialized productivity functions?

3. Would a hybrid interaction model combining hand tracking and AI-powered voice control be acceptable under the competition rules?

4. Are there any specific recommendations for demonstrating accessibility, usability, and compliance with the hand-tracking requirements for enterprise-oriented digital twin applications?

    

We believe that combining immersive 3D digital twins, multilingual AI voice interaction, and hand tracking can create a valuable new productivity experience for Meta VR Glasses users.

We have attached an example image illustrating our multilingual AI avatar interaction and VR soccer simulation i

nterface.

We would greatly appreciate your guidance before finalizing our competition submission.
 

Best regards,

Abocado