Forum Discussion
vivekrajagopal
3 years agoMHCP Member
Meta Avatars 2 Lipsync - PUN 2 & Photon Voice issue!
Dear Devs,
I'm struggling with problem since a week. I use PUN 2 and Photon Voice to bring Meta Avatars 2 in a multiplayer environment.
Below are my issues,
1. When there is no Photon Voice setup in the scene, the Meta Avatars lipsync works perfect in the Photon multiplayer room.
2. When I add the Photon Voice to the prefab and setup the scene with Photon Voice Network, only the voice is there the Meta Avatars lipsync does not work.
I understand there is a race condition happening between these two plugins.
Please kindly help me resolve if anyone has already resolved such problem. This thread can help other devs as well in the future.
Thanks!
Dear Developers,
I've found a proper fix for this issue. I've explained it in my blog that might help you.
Please check this blog: https://medium.com/@vivekrajagopal_84414/meta-avatars-2-and-photon-unity-voice-issue-lipsync-pun-voice-resolved-6a7c5d229b1c
Dear Developers,
Hi, I've found a proper fix for this issue. I've explained it in my blog that might help you.
Please check this blog: https://medium.com/@vivekrajagopal_84414/meta-avatars-2-and-photon-unity-voice-issue-lipsync-pun-voice-resolved-6a7c5d229b1c
29 Replies
Replies have been turned off for this discussion
- ezoneProtege
Sorry, I couldn't find a workaround either, so I ended up disabling lip-sync. Ideally there would be a checkbox on the avatar system to allow PUN to also access the microphone.
- olaysiaExpert Protege
I found a solution to this:
The Problem:
The scripts UnityMicrophone(by Photon) and LipSyncMicInput(part of the avatars 2 sdk) both contain this line
Microphone.Start(deviceName, looping, audioClipLength, samplingRate).This method creates a new audio clip which gets filled with the input from your microphone. This causes the following situation:
- UnityMicrophone creates a new audio clip for itself using the Microphone.Start() function and is happily using an audio clip which is being continuously filled with the input of your microphone.
- Now LipSyncMicInput also calls the Microphone.Start(), now the input of your microphone is being stolen by LipSyncMicInput's audio clip and the audio clip which Photon's UnityMicrophone was using now helplessly loops over itself.
- The same can happen the other way around, it just depends who called Microphone.Start() first.
In summary, the audio clip which LipSyncMicInput or Photon.Recorder is using will be useless as soon as one of them calls Microphone.Start().
My Solution:
- I created a central Microphone object which would have the sole responsibility of Starting and stopping the Microphone.
- To deal with Photon, I changed the Input source from Microphone to Factory. And created a custom input source which used my central microphone as its input.
- To deal with LipSync, I REMOVED LipSyncMicInput from my LipSyncInput object, and replaced it with a script which would continuously check the LipSyncInput object's audio source clip against my central Microphone object's audio clip. If different, it would update the lip sync's audio clip to be the same as my central Microphone's. I also had to force the clip's position to match that of my microphone's.
For help creating a custom input source for the Photon Recorder I recommend just googling it.
Here's the script I used to replace LipSyncMicInput, GGMicrophone is my central microphone object.
public class RuntimeAudioClipUpdater : MonoBehaviour { AudioSource audioSource; private void Start() { audioSource = GetComponent<AudioSource>(); } void Update() { if (audioSource != null) { AudioClip clip = audioSource.clip; AudioClip referenceClip = GGMicrophone.Instance.GetMicrophoneAudioClip(); int referenceClipId = referenceClip.GetInstanceID(); if (clip != null) { int currentClipId = clip.GetInstanceID(); if (currentClipId != referenceClipId) { //audioSource.Stop(); audioSource.clip = referenceClip; audioSource.Play(); } } else { //audioSource.Stop(); audioSource.clip = referenceClip; audioSource.Play(); } } audioSource.timeSamples = GGMicrophone.Instance.GetPosition(); } }- ErickMarinCStart PartnerHi Olaysia, that is a great solution! Thanks for sharing.I have been struggling to use a custom audio source with PUN voice 2 Recorder, based on the documentation.Is there any chance to share your implementation.Thank you!
- olaysiaExpert Protege
Sure, which part would you like me to share? The Central Microphone, the custom photon recorder input or the lips sync clip updater?
- EverysimoExplorer
Hi, i'm trying to do as you did with your code but it keeps giving me the
NullReferenceExeprion: Object reference not set to an istance of an object
RuntimeAudioClipUpdater.Update() (RuntimeAudioClipUpdater.cs.20).
What i did is:
1) Changing the Recorder Microphone Type to Factory
2) (LipSyncInput) Removing Lip Sync Mic Input and setting Ovr Avataer Lip Sync Context
3) (LipSyncInput) Adding Runtime Audio Clip Updater
4) Cretaing a new empty object than call GGMicrophoneInputForPhotonInitializer
Am i missing something, or doing something wrong? The script are the same as the one you pubblished- olaysiaExpert Protege
You may have to tweak things a bit for your particular solution. But let me see if I can help. If you click on the null reference exception, which line of the script RuntimeAudioClipUpdater is it complaining about?
- EverysimoExplorer
Hi, sorry i forgot to share the line of code
AudioClip referenceClip = GGMicrophone.Instance.GetMicrophoneAudioClip();
- vivekrajagopalMHCP Member
Dear Developers,
I've found a proper fix for this issue. I've explained it in my blog that might help you.
Please check this blog: https://medium.com/@vivekrajagopal_84414/meta-avatars-2-and-photon-unity-voice-issue-lipsync-pun-voice-resolved-6a7c5d229b1c
- olaysiaExpert Protege
Nice! That's a better solution to what I've got.
- violinviolin19Explorer
Could you share your sample repo please?
- emalafeewProtege
Thank you incredibly much for this post!
- vivekrajagopalMHCP Member
Glad I could help.
- RanggitchuHonored Guest
Nice
- RanggitchuHonored Guest
Good
Quick Links
- Horizon Developer Support
- Quest User Forums
- Troubleshooting Forum for problems with a game or app
- Quest Support for problems with your device
Other Meta Support
Related Content
- 9 months ago
- 4 months ago
- 2 months ago
- 10 months ago