Forum Discussion

vivekrajagopal's avatar
vivekrajagopal
Protege
4 years ago
Solved

Meta Avatars 2 Lipsync - PUN 2 & Photon Voice issue!

Dear Devs,   I'm struggling with problem since a week. I use PUN 2 and Photon Voice to bring Meta Avatars 2 in a multiplayer environment.  Below are my issues, 1. When there is no Photon Voice se...
Lipsync
Meta Avatars
Meta Avatars 2
Photon
PUN 2
Unity
violinviolin19's avatar
violinviolin19
Explorer
3 years ago

Hi olaysia, thanks so much for the information! This has really helped my project. Would you also be able to share the lip sync clip updater? I am not able to get mine to work properly.

olaysia's avatar
olaysia
Expert Protege
3 years ago
violinviolin19 wrote:

Hi olaysia, thanks so much for the information! This has really helped my project. Would you also be able to share the lip sync clip updater? I am not able to get mine to work properly.

Sure! Firstly let me describe some of my additional setup:

  1. Find the LipSyncInput gameobject in your scene.
  2. Remove LipSyncMicInput.
  3. Set OvrAvatarLipSyncContext to use the AudioSource.
  4. Add this RuntimeAudioClipUpdater to the object.

It's purpose is to check if your central microphone's audio clip has changed. If it has, it replaces the AudioSource's clip to the same as that on your central microphone.

 

public class RuntimeAudioClipUpdater : MonoBehaviour
{
    AudioSource audioSource;

    private void Start()
    {
        audioSource = GetComponent<AudioSource>();
    }

    void Update()
    {
        if (audioSource != null)
        {
            AudioClip clip = audioSource.clip;
            AudioClip referenceClip = GGMicrophone.Instance.GetMicrophoneAudioClip();
            if (referenceClip != null)
            {
                int referenceClipId = referenceClip.GetInstanceID();
                if (clip != null)
                {
                    int currentClipId = clip.GetInstanceID();
                    if (currentClipId != referenceClipId)
                    {
                        //audioSource.Stop();
                        audioSource.clip = referenceClip;
                        audioSource.Play();
                    }
                }
                else
                {
                    //audioSource.Stop();
                    audioSource.clip = referenceClip;
                    audioSource.Play();
                }
                audioSource.timeSamples = GGMicrophone.Instance.GetPosition();
            }
        }
    }


}

 

 

  • Everysimo's avatar
    Everysimo
    Explorer
    3 years ago

    Hi, i'm trying to do as you did with your code but it keeps giving me the
    NullReferenceExeprion: Object reference not set to an istance of an object 
    RuntimeAudioClipUpdater.Update() (RuntimeAudioClipUpdater.cs.20).

     

     



    What i did is:
    1) Changing the Recorder Microphone Type to Factory

     


    2) (LipSyncInput) Removing Lip Sync Mic Input and setting Ovr Avataer Lip Sync Context

     


    3) (LipSyncInput) Adding Runtime Audio Clip Updater
    4) Cretaing a new empty object than call GGMicrophoneInputForPhotonInitializer

     



    Am i missing something, or doing something wrong? The script are the same as the one you pubblished

  • violinviolin19's avatar
    violinviolin19
    Explorer
    3 years ago

    Hi olaysia, Thanks so much for your help, this works like a charm! Recently, I've been working with custom lip sync data instead of input through a microphone, so I decided to repurpose this script to get the lip sync working. I was able to get the lip sync action occurring, but it doesn't quite match up with the words and it loops one more time before stopping even though the loop feature is disabled. I think it has to do with the last line, because I don't know what the equivalent to setting the timeSamples variable would be in my script. 

    using Meta.WitAi.TTS.Utilities;
using UnityEngine;

public class RuntimeAudioClipUpdater : MonoBehaviour
{
    AudioSource audioSource;
    [SerializeField] TTSSpeaker speaker;

    private void Start()
    {
        audioSource = GetComponent<AudioSource>();
    }

    void Update()
    {
        if (audioSource != null)
        {
            AudioClip clip = audioSource.clip;
            AudioClip referenceClip = null;
            if (speaker.SpeakingClip != null)
            {
                referenceClip = speaker.SpeakingClip.clip;
            }
            if (clip != null && referenceClip != null)
            {
                int referenceClipId = referenceClip.GetInstanceID();
                int currentClipId = clip.GetInstanceID();
                if (currentClipId != referenceClipId)
                {
                    //audioSource.Stop();
                    audioSource.clip = referenceClip;
                    audioSource.Play();
                }
            }
            else if(referenceClip != null)
            {
                //audioSource.Stop();
                audioSource.clip = referenceClip;
                audioSource.Play();
            }

        }
        //audioSource.timeSamples = speaker.AudioSource.timeSamples;
    }


}

    Here's what my script looks like right now. As you can see, I tried to set the audioSource timeSamples to be equal to the reference clip timeSamples that I'm sending in, but that's just redundant and doesn't work. Have you tried this type of thing before or have any insights on how to solve this problem? 

    • olaysia's avatar
      olaysia
      Expert Protege
      3 years ago

      Hi ViolinViolin19,

      So I actually ended up scrapping my solution in favour of Vivek's solution which you can see further down in the replies. Here's solution resolves both of the problems you are having.