Forum Discussion
OVR LipSync Open Source
Meta created a great lip sync tool with OVR LipSync. Looking at how Meta is slowly becoming the Open Source leader, I was wondering if they would ever make this plugin Open Source and let the community develop it further. It seems stuck in limbo. Saw this video on how they initially had plans to even bring it to iOS and MacOS here . I feel it has so much more potential then only being stuck on headsets.
What do you guys think? Do you think we can encourage Meta to Open Source OVR Lipsync?
4 Replies
Replies have been turned off for this discussion
This is would be a very productive move. The OVRLipSync library dates back to 2015 and doesn't get any active support.
It's still not got a Linux version! https://communityforums.atmeta.com/t5/Unreal-VR-Development/OVR-Lip-Sync-plugin-for-Linux/td-p/971870
Releasing the source code would allow the community to fill this and other important gaps that are not being prioritized.
Additionally, the specific set of visemes generated by this library has become the de facto standard for all avatars in the absence of any alternative, so a lot of assets depend on it.
I've posted a similar issue in the Godot forums https://github.com/godotengine/godot-proposals/discussions/9718
This move should be consistent with their strategy of support for the Godot platform in terms of its capability to deploy apps on the Quest.
https://www.w4games.com/blog/w4-games-news-1/w4-games-and-meta-team-up-to-expand-open-source-godot-ecosystem-on-meta-quest-28Now how to get Meta's attention on this. I honestly think they forgot they made this tool, which is why it's collecting dust.
I've compiled it into a demo Android app https://github.com/goatchurchprime/lipsync through a Godot addon called twovoip_lipsync, so we can use it in Godot now (except on Linux) to see if we can do anything useful with it. Maybe this will get some attention.
Meanwhile, it's worth keeping an eye on some tech adjacent projects, such as https://github.com/Project-Babble/ for face tracking that seems to use onnxruntime ai tech. Given that the https://github.com/xiph/rnnoise library I have compiled into twovoip also uses ai tech to filter out non-speech noise, it is likely that someone who knows this tech could replicate the OVRLipSync library using modern ai tech better than before.
I noticed OVRLipsync on both Unity and Unreal has an End of Life Notice. It says it's subject to removal. Please Meta! Now is the time to release the source code and model! Open Source all the way before OVRLipSync dies!
