Skip to main content
Ascensi
Up and Coming
August 27, 2014

Facial Mocap

We need (or at least I need) a facial mocap system that be compatible with or integrated with Oculus/HMDs

I believe this would be a normal evolution of VR HMDs because gamers, game developers/mocap developers for film and games could benefit this in real-time providing instant feedback. I think this could be done without much additional cost if the Rift mask edge has small EEG type electrodes/thermal sense etc to track facial expression built into it or detects the shape of the mask becoming altered. Eye tracking would have to be in here somehow as well.

It would be more immersive that when you meet people/ your friends in various worlds that they experience something other than preprogrammed game character expressions and react familiar to normal conversation.

Just an idea.. that OR or maybe someone else could invent ;)

    4 replies

    zalo
    Up and Coming
    August 28, 2014
    Now that leap has unveiled their Image API, we can mount it in fun places like underneath the rift (Here it's on the DK1's under lip):



    Granted, it will take some mildly complex image processing to determine phoneme and expression, but it's doable.

    The distortion on the lenses coupled with the close-up nature of the shot makes it look a little strange to our human sensibilities!
    saviornt
    Up and Coming
    August 29, 2014
    There are already apps out there that do this, however, the price point is geard towards AAA animation studios.

    It would rock if Sony open-sourced what they developed for the EQ franchise. Its not precise, but it is good enough for small-timers
    Current WIPs using Unreal Engine 4: Agrona - Tales of an Era : Medieval Fantasy MORPG
    mptp
    Up and Coming
    August 31, 2014
    Go and look into High Fidelity - decent facial tracking using a standard we can is going to be possible very soon for consumers. The only issue is its not possible to capture facial tracking data from someone wearing a HMD right now...
    Melbourne-based creative technologist. I flit between experimental AR/VR experiences, audiovisual electronics and full-stack web development. http://www.lachansleight.io
    ganzuul
    Up and Coming
    September 17, 2014
    From what I've seen of Star Citizen mocap they are having a problem with very stiff lips.

    The video that zalo posted might be a way to address this. However, presumably gamers will keep wanting to snack on junk food while gaming.

    Might have to resort to projecting IR lasers in people's faces.
    Terms & ConditionsAccessibility statement