Forum Discussion
Facial Mocap
We need (or at least I need) a facial mocap system that be compatible with or integrated with Oculus/HMDs
I believe this would be a normal evolution of VR HMDs because gamers, game developers/mocap developers for film and games could benefit this in real-time providing instant feedback. I think this could be done without much additional cost if the Rift mask edge has small EEG type electrodes/thermal sense etc to track facial expression built into it or detects the shape of the mask becoming altered. Eye tracking would have to be in here somehow as well.
It would be more immersive that when you meet people/ your friends in various worlds that they experience something other than preprogrammed game character expressions and react familiar to normal conversation.
Just an idea.. that OR or maybe someone else could invent ;)
I believe this would be a normal evolution of VR HMDs because gamers, game developers/mocap developers for film and games could benefit this in real-time providing instant feedback. I think this could be done without much additional cost if the Rift mask edge has small EEG type electrodes/thermal sense etc to track facial expression built into it or detects the shape of the mask becoming altered. Eye tracking would have to be in here somehow as well.
It would be more immersive that when you meet people/ your friends in various worlds that they experience something other than preprogrammed game character expressions and react familiar to normal conversation.
Just an idea.. that OR or maybe someone else could invent ;)
4 Replies
- Now that leap has unveiled their Image API, we can mount it in fun places like underneath the rift (Here it's on the DK1's under lip):
Granted, it will take some mildly complex image processing to determine phoneme and expression, but it's doable.
The distortion on the lenses coupled with the close-up nature of the shot makes it look a little strange to our human sensibilities! - There are already apps out there that do this, however, the price point is geard towards AAA animation studios.
It would rock if Sony open-sourced what they developed for the EQ franchise. Its not precise, but it is good enough for small-timers - Go and look into High Fidelity - decent facial tracking using a standard we can is going to be possible very soon for consumers. The only issue is its not possible to capture facial tracking data from someone wearing a HMD right now...
- From what I've seen of Star Citizen mocap they are having a problem with very stiff lips.
The video that zalo posted might be a way to address this. However, presumably gamers will keep wanting to snack on junk food while gaming.
Might have to resort to projecting IR lasers in people's faces.
