Cool MR Projects Part 1: Back to the Mixed Reality!

 Welcome to the first part of our series, Cool MR Projects!

In this post, I will review my own Mixed Reality (MR) project, "Back to the Mixed Reality," a fan-made experience that demonstrates the potential of MR. This project allows users to virtually drive the iconic DeLorean Time Machine and experience time travel right on their floor.

Mixed Reality (MR) involves the blending of physical and digital worlds, creating new environments where physical and digital objects co-exist and interact in real time. If you've ever wondered why I built this project and what challenges we faced, read on!

The Motivation: Why Build a Mixed Reality Time Machine?

The inspiration for "Back to the Mixed Reality" stems directly from childhood dreams. I have always been fascinated by the concept of time travel, specifically from the Back to the Future movie series. As a child, I imagined time-traveling on my bicycle, believing that if I reached a certain speed, I would travel to the time I was dreaming of. Now, using advancements in Mixed Reality with Meta Quest 3, I’ve been able to turn this childhood fantasy into a real experience.

Beyond personal passion, the project serves several key goals:

1. Demonstrating MR Potential: This project showcases the incredible potential of Mixed Reality to bring childhood dreams to life.
2. Inspiring the Community: It is a passion project born from a love for the Back to the Future series and a fascination with MR's potential. The project is designed to inspire, entertain, and educate enthusiasts and developers about the magic of MR technology.
3. Driving Attention to Spatial Computing: My biggest intention in building this fan-made project is to drive more and more attention to Mixed Reality, Virtual Reality, or spatial computing.

The experience involves spawning the DeLorean Time Machine on your floor, setting a destination time (e.g., 20 seconds into the future), driving the vehicle to reach the required time travel speed, and then waiting for it to return from the future.

The Challenges: Navigating the New Frontier of MR Development

The core mechanics required detailed visual effects (VFX) using Unity Timelines, Particles, and Shaders to replicate the time travel sequence from the first movie. We also used FMOD to create adaptive car sound effects that change according to the speed and RPM of the car engine, integrating them into Unity using the FMOD Unity plugin.

However, integrating Mixed Reality functionality introduced several unique hurdles:

1. Model Optimization

After importing the detailed 3D models of the iconic DeLorean Time Machine and the remote controller (inspired by Doc’s remote in the first movie) into Unity, we had to perform heavy optimizations because the models were "insanely detailed".

2. Testing and Iteration Time

Integrating MR functionality brought new challenges and design considerations. Testing and iterating with Room Setup was very time-consuming because every time a change was made, it required testing within the headset.

Fortunately, this challenge was mitigated by discovering the Meta XR Simulator. This tool literally saved hours of time, allowing me to test the XR project without constantly wearing the headset by simulating headset movement and touch controller input using a keyboard, mouse, or game controller.

3. Ensuring Realistic Collisions and Boundaries

A crucial challenge was ensuring the virtual car remained within the room's boundaries and had realistic collisions with the physical floor and real-life objects.

To achieve this, we relied on Scene Understanding—a cutting-edge technology providing a comprehensive Scene Model for geometric and semantic representation of the physical space. We had to:

• Identify room elements (like Floor, ceiling, Wall Face, etc.) using the OVRSemanticClassification class.
• Add colliders to the room's walls and floors, which required careful adjustment and testing.
• Tag the corresponding colliders based on the type of room element to ensure the car properly collided with real-world objects.

4. Managing the App Flow and Room Setup

For Mixed Reality experiences using Meta Quest 3, the user must proceed with initial settings before loading the experience. The experience will not work without the user performing a Room Setup.

We created a Lobby Scene to manage these MR procedures:

• The user must first give permission to access spatial data, which is necessary for the app to utilize Scene Understanding.
• If the user has not completed the Room Setup, they are prompted to do so before the game scene is loaded.
• We utilized knowledge gleaned from Project Phanto (a Unity-based Mixed Reality reference app from Meta demonstrating critical features like scene mesh and general app flow) to build the general app flow. This included informing the user about Scene Mesh visualization once the setup is complete.

Join the Adventure!

With all these mechanics, I successfully built a working Time Machine. I hope this project drives more attention to the magic of Mixed Reality.

If you want to play the experience on your own Quest 3, you can get the build for FREE! 

You can also watch the deep dive YouTube video here!

Let's drive into the future, together!

– Tevfik (Creator of Back to the Mixed Reality)