cancel
Showing results for 
Search instead for 
Did you mean: 

Full body tracking through Oculus runtime?

cameron.kostopoulos
Honored Guest

I am building a VR installation that demands full body tracking. Currently it's built with Unreal Engine and uses Vive Trackers and an IK system for the MOCAP. However, I'm hoping to incorporate face and hand tracking from the Quest Pro, in combination with the full body tracking, for even more immersion. This raises a lot of technical issues and I was wondering if someone here could shed some light...

1. Is it possible for Oculus runtime to access Vive trackers? Or is this strictly SteamVR? And if it is SteamVR, is it possible to have two runtimes communicating with each other or is this impossible?

2. So if the Vive trackers are impossible... What full body solutions exist for Oculus runtime? Is it possible to use something like SlimeVR (I read that this was also SteamVR). Would a MOCAP suit, like Perception Neuron, work? Is there another solution I'm missing?

In short... How the heck do you get a full body capture, including hand and face, on the Quest?

1 ACCEPTED SOLUTION

Accepted Solutions

BattleAxeVR
Expert Protege

If you want face tracking and eye tracking, you'll need a Quest Pro. It works really well, btw. Eye-tracking is awesome (yes, it works on PC too).

For hand/body tracking on Quest, the best solution to reach the most customers is the new OpenXR body tracking extension (returns articulated finger poses from hand tracking. this is the only way on PC right now).

It works on PC via Link/Airlink too, on all Quests (including 1). I used the hip joint pose to create a demo of waist-oriented locomotion here. Full source included and pre-built exe and APKs for testing also:

https://forums.oculusvr.com/t5/OpenXR-Development/Waist-Oriented-locomotion-demo/td-p/1002044

Legs aren't returned yet but I believe Meta is working on that. The farther you get away from the HMD and 2 controllers (or hands), the more error-prone it is and harder it is to deliver a good synthetic thigh / knee / angle / foot joint estimate. But I hope they can deliver that too. TBH putting trackers on is always going to be very niche, Quest's solution is better.

View solution in original post

3 REPLIES 3

BattleAxeVR
Expert Protege

If you want face tracking and eye tracking, you'll need a Quest Pro. It works really well, btw. Eye-tracking is awesome (yes, it works on PC too).

For hand/body tracking on Quest, the best solution to reach the most customers is the new OpenXR body tracking extension (returns articulated finger poses from hand tracking. this is the only way on PC right now).

It works on PC via Link/Airlink too, on all Quests (including 1). I used the hip joint pose to create a demo of waist-oriented locomotion here. Full source included and pre-built exe and APKs for testing also:

https://forums.oculusvr.com/t5/OpenXR-Development/Waist-Oriented-locomotion-demo/td-p/1002044

Legs aren't returned yet but I believe Meta is working on that. The farther you get away from the HMD and 2 controllers (or hands), the more error-prone it is and harder it is to deliver a good synthetic thigh / knee / angle / foot joint estimate. But I hope they can deliver that too. TBH putting trackers on is always going to be very niche, Quest's solution is better.

Thank you for the detailed info!! I totally agree that putting on trackers is niche and will hopefully be fully replaced soon. QuestSim looks super cool and I hope it’s made available soon. Unfortunately for this specific use case an accurate full body capture is super important, so we’ve opted for a Vive Pro Eye + facial tracker + LeapMotion for hands. Hopefully legs come to the Quest soon because this is such a niche hardware combo that it’ll really only work for installations, but at this stage it’s worth the trade off.

Totally, I just think, if anything, Quest's "fake FBT" will permit many devs to justify the time / energy / expense of bothering to support FBT in their games, which is available to all as a core platform feature, not as an add-on that only a few people will own, which in turn will benefit real FBT systems because there will be more software to take advantage of FBT (real or fake). That's the way I see it, anyway. So, have at it!

Btw I also think it's possible for Quest 1 and 2 to support hand tracking at the same time as the Touch Pro controllers which track themselves inside-out. So you can use one free hand and 1 controller do use buttons and sticks for navigation, but the other hand to manipulate objects, etc. It's also easier to get into and out of VR with only 1 controller to worry about, especially when you add hand straps. For noobs, I mean, it is a nuisance that requires outside assistance usually.