We are creating a VR multiuser platform using Ready Player Me avatars and Photon Realtime for networking. We had added lipsyncing to the avatars which is working fine with up to 10 to 12 users. After that, the sound quality goes poor due to too much crackling in the sound. When we turned off lipsync, voice quality improved and so did CPU usage. We need to optimize lipsync in ways that we can turn it on again without hurting CPU usage and sound quality. Please suggest how can we do it? Can turning off a few viseme or keeping their weights low help us? What measures can we take to optimize it?
