Forum Discussion

xlilx3020's avatar
xlilx3020
Honored Guest
28 days ago

I cannot use VisemeDriver Component

I would like to estimate visemes from audio in Unity.
I configured my project according to the documentation, but FaceExpression.AreVisemesValid is always false, and as a result, the VisemeDriver component does not work.


This issue also occurs in the Face Tracking sample scene included with the Meta Movement SDK.

Unity version: 6000.2.10f1
Meta Core SDK version: 81.0.0
Meta Movement SDK version: 81.0.0

How can I use the VisemeDriver?
Please let me know if there is a workaround or solution.

3 Replies

  • In the Face Tracking sample scene of the Movement SDK, it can be confirmed that lip sync is working based on audio input. However, this does not provide viseme labels. Instead, it simply derives the weights of OVRFaceExpressions.FaceExpression from the audio and updates the corresponding blend shapes.

    This approach is inconvenient and cannot be used when an avatar’s blend shapes are authored to correspond specifically to visemes.

    Urgent action is required to address this issue.
    How are others handling this situation?

  • AreVisemesValid is updated inside the Update function in OVRFaceExpressions.cs at line 213.
    It is computed as the logical AND of the following two conditions:

    • OVRPlugin.GetFaceVisemesState(OVRPlugin.Step.Render, ref _currentFaceVisemesState) == OVRPlugin.Result.Success
    • _currentFaceVisemesState.IsValid

    The first function, OVRPlugin.GetFaceVisemesState, is defined in OVRPlugin.cs at line 10308.
    Inside it, the following call at line 10315 returns Failure_Unsupported:

    Result res = OVRP_1_104_0.ovrp_GetFaceVisemesState( stepId, -1 /* OVRP_CURRENT_FRAMEINDEX */, out cachedFaceVisemesState );

    Because this call returns Failure_Unsupported, AreVisemesValid always becomes false.

    Since OVRP_1_104_0.ovrp_GetFaceVisemesState is implemented inside a DLL, I cannot inspect its internal implementation.

    Although the Meta Movement SDK claims to support visemes, the current situation is that they cannot actually be used. This appears to require a fix. I am looking for an alternative approach.

    Please help.

→ Find helpful resources to begin your development journey in Getting Started

→ Get the latest information about HorizonOS development in News & Announcements.

→ Access Start program mentor videos and share knowledge, tutorials, and videos in Community Resources.

→ Get support or provide help in Questions & Discussions.

→ Show off your work in What I’m Building to get feedback and find playtesters.

→ Looking for documentation?  Developer Docs

→ Looking for account support?  Support Center

→ Looking for the previous forum?  Forum Archive

→ Looking to join the Start program? Apply here.

 

Recent Discussions