Forum Discussion
Lipsync From Other Means
Hello there, I am new to the forum, and working on a project currently for my Final Year Project in college. I am trying to build a virtual assistant which maps the output from a dialog system to the...
Hi,
My project is on Oculus Quest and while we have realtime multiplayer VOIP working, I found that the CPU cost for doing LipSync from VOIP data was simply too expensive on Quest.
I switched to "canned" LipSync using precomputed viseme data (essentially a list of float values). I found the Unity LipSync sample application more useful for me than the UE4 version - as it has a tool to generate viseme data from an audio asset as well as a sample precomputed viseme data file that I ended up adding to my UE4 code.
I switched to "canned" LipSync using precomputed viseme data (essentially a list of float values). I found the Unity LipSync sample application more useful for me than the UE4 version - as it has a tool to generate viseme data from an audio asset as well as a sample precomputed viseme data file that I ended up adding to my UE4 code.
So what you're trying to do should be possible on Rift hardware, provided that you can get your audio data as raw PCM bytes that you can feed to the LipSync system. See UOVRLipSyncActorComponent::FeedAudio for how this is done.