cancel
Showing results for 
Search instead for 
Did you mean: 

Meta Avatars 2 Lipsync - PUN 2 & Photon Voice issue!

vivekrajagopal
Protege

Dear Devs,

 

I'm struggling with problem since a week. I use PUN 2 and Photon Voice to bring Meta Avatars 2 in a multiplayer environment. 

Below are my issues,

1. When there is no Photon Voice setup in the scene, the Meta Avatars lipsync works perfect in the Photon multiplayer room.

2. When I add the Photon Voice to the prefab and setup the scene with Photon Voice Network, only the voice is there the Meta Avatars lipsync does not work. 

 

I understand there is a race condition happening between these two plugins. 

 

Please kindly help me resolve if anyone has already resolved such problem. This thread can help other devs as well in the future.

 

Thanks!

27 REPLIES 27

vivekrajagopal
Protege

Dear Developers, 

I've found a proper fix for this issue. I've explained it in my blog that might help you. 

Please check this blog: https://medium.com/@vivekrajagopal_84414/meta-avatars-2-and-photon-unity-voice-issue-lipsync-pun-voi...

Dear Developers, 

Hi, I've found a proper fix for this issue. I've explained it in my blog that might help you. 

Please check this blog: https://medium.com/@vivekrajagopal_84414/meta-avatars-2-and-photon-unity-voice-issue-lipsync-pun-voi...

Nice! That's a better solution to what I've got.

Hi olaysia, Thanks so much for your help, this works like a charm! Recently, I've been working with custom lip sync data instead of input through a microphone, so I decided to repurpose this script to get the lip sync working. I was able to get the lip sync action occurring, but it doesn't quite match up with the words and it loops one more time before stopping even though the loop feature is disabled. I think it has to do with the last line, because I don't know what the equivalent to setting the timeSamples variable would be in my script. 

using Meta.WitAi.TTS.Utilities;
using UnityEngine;

public class RuntimeAudioClipUpdater : MonoBehaviour
{
    AudioSource audioSource;
    [SerializeField] TTSSpeaker speaker;

    private void Start()
    {
        audioSource = GetComponent<AudioSource>();
    }

    void Update()
    {
        if (audioSource != null)
        {
            AudioClip clip = audioSource.clip;
            AudioClip referenceClip = null;
            if (speaker.SpeakingClip != null)
            {
                referenceClip = speaker.SpeakingClip.clip;
            }
            if (clip != null && referenceClip != null)
            {
                int referenceClipId = referenceClip.GetInstanceID();
                int currentClipId = clip.GetInstanceID();
                if (currentClipId != referenceClipId)
                {
                    //audioSource.Stop();
                    audioSource.clip = referenceClip;
                    audioSource.Play();
                }
            }
            else if(referenceClip != null)
            {
                //audioSource.Stop();
                audioSource.clip = referenceClip;
                audioSource.Play();
            }

        }
        //audioSource.timeSamples = speaker.AudioSource.timeSamples;
    }


}

Here's what my script looks like right now. As you can see, I tried to set the audioSource timeSamples to be equal to the reference clip timeSamples that I'm sending in, but that's just redundant and doesn't work. Have you tried this type of thing before or have any insights on how to solve this problem? 

Hi ViolinViolin19,

So I actually ended up scrapping my solution in favour of Vivek's solution which you can see further down in the replies. Here's solution resolves both of the problems you are having.

Could you share your sample repo please? 

Thank you incredibly much for this post!

 

Glad I could help.