Forum Discussion
OVR LipSync Bug: Multiple targets mouth the same?
Hi,
I have a scene with multiple users using OVR Lip Sync.
However, if one user talks *all* users mouth movements are sync'd to that (on their local game).
If I debug out OVRLipSyncContext, they each have a unique context id, so it should keep them separated during decoding. however, it doesn't appear to work internally.
It is easy to repro using the OVR Lip Sync install:
1. Create a new scene
2. Drag in a "LipSyncInterface" prefab
3. Drag in "LipSyncMorphTarget_RobotTexture" prefab
3.1 Name it "Audio File", and delete the OVRLipSyncMicInput, enable OVRLipSyncTestAudio, drag on "OVRLipSync/Audio/vox_lp_01" onto the audio source
4. Drag in a second "LipSyncMorphTarget_RobotTexture" prefab
4.1 rename it "Mic" . Duplicate the material in OVRLipSyncContextTextureFlip so it is different.
Press PLAY
Expected:
The "Audio File' keeps mouthing that file while the "Mic" plays what I say
Actual:
The "Audio File" stops mouthing, and both mouth whatever I say.
I am on Unity 5.2.3p1 with OVR Lip Sync V1.0.1-beta.
Am I missing something obvious?
I have a scene with multiple users using OVR Lip Sync.
However, if one user talks *all* users mouth movements are sync'd to that (on their local game).
If I debug out OVRLipSyncContext, they each have a unique context id, so it should keep them separated during decoding. however, it doesn't appear to work internally.
It is easy to repro using the OVR Lip Sync install:
1. Create a new scene
2. Drag in a "LipSyncInterface" prefab
3. Drag in "LipSyncMorphTarget_RobotTexture" prefab
3.1 Name it "Audio File", and delete the OVRLipSyncMicInput, enable OVRLipSyncTestAudio, drag on "OVRLipSync/Audio/vox_lp_01" onto the audio source
4. Drag in a second "LipSyncMorphTarget_RobotTexture" prefab
4.1 rename it "Mic" . Duplicate the material in OVRLipSyncContextTextureFlip so it is different.
Press PLAY
Expected:
The "Audio File' keeps mouthing that file while the "Mic" plays what I say
Actual:
The "Audio File" stops mouthing, and both mouth whatever I say.
I am on Unity 5.2.3p1 with OVR Lip Sync V1.0.1-beta.
Am I missing something obvious?
7 Replies
Replies have been turned off for this discussion
- Ahh, found the issue.
It is because the way OVRLipSyncContext works is it has a material for the mouth textures which has to match the material on the mouth renderer.
Now, when you instantiate a prefab of it, obviously all objects have the same material so any change to one effects the other.
What I failed to notice at the time was I didn't change both materials.
The fix I used was to change the OVRLipSyncContext to not use the material but rather just access the instanced material on the mouth object with public Renderer MouthRenderer;
MouthRenderer.material instead of material.
Now it works great. - If you are using Lip Sync with texture flip in a prefab, obviously you can edit OVRLipSyncContextTextureFlip.cs to access a new material. But if you want to avoid changing the Oculus scripts, here is a script that will do it for you. Place it on the same game object as the OVRLipSyncContextTextureFlip.
It serves 2 purposes:
1. Sets the material to point to the mouth material
2. By accessing the Render.material at runtime it creates a new material instance so it is no longer shared between prefabs
SeetupOVRLipSyncContextTextureFlip.cs:using UnityEngine;
using System.Collections;
[RequireComponent(typeof(OVRLipSyncContextTextureFlip))]
public class SeetupOVRLipSyncContextTextureFlip : MonoBehaviour
{
public Renderer MouthRenderer;
void Awake()
{
GetComponent<OVRLipSyncContextTextureFlip> ().material = MouthRenderer.material;
}
}
HTH someone else!
I suggest in the future Oculus could change OVRLipSyncContextTextureFlip to take a reference to the renderer so it does this automatically as I expect most games are going to have this in a prefab. - Thank you for finding this! We will fix our script to reference an instanced version of the material.
- I have a fix that I will add the the OVRLipSyncTextureFlip script.
You can add this bit of code at the end of the Start function, which will copy the material to a temp location,create a new material list and copy the temp back:
// Instantiate the material that is set in this script to each renderer in the children
Renderer [] renderers = GetComponentsInChildren<Renderer>();
foreach (Renderer r in renderers)
{
if (r.sharedMaterial == material)
{
// Copy mat to temp location
Material [] tmp = new Material[1];
tmp [0] = new Material (r.sharedMaterial);
// Create a new material list and re-copy back
r.materials = new Material[1];
r.materials [0].CopyPropertiesFromMaterial(tmp[0]);
material = r.materials [0];
}
}
This should fix the issue. "petergiokaris" wrote:
Why is tmp an array of size 1 and not just?
Material [] tmp = new Material[1];
tmp [0] = new Material (r.sharedMaterial);
Material tmp = new Material (r.sharedMaterial);
Was this just to match the style of the existing code or is there some kind of "this will create less garbage for the GC" optimisation I am not realizing that is important to do in Unity.- This was to match the style of the existing code that I found from Unity forums.
- Thanks Peter - I can confirm your fix works!
In the context of a multi-user scenario, I assume it's quite a heavy operation to decode the audio stream, is there any way to decode the visemes only on the local users device, and then transmit them with voice data packets (to avoid all client devices decoding multiple audio streams) ?
Peter
