Oculus Avatars Expressive Update - Unity Preview. 02.20.2019

[Ross_Beef]

CONFIDENTIAL
We appreciate your discretion during this early access period.
Please don't discuss these updates or share publicly any images or video of expressive Avatars in action.
We'd love your feedback. Please keep all comments / discussions regarding bugs or work in progress with expressive avatars restricted to the Avatar SDK Developer Preview discussion, and avoid posting to the broader public avatar discussion thread.


Hey folks, 

Here's the first drop (2019.02.20) of the expressive avatar update, which introduces eye gaze, lipsync and facial microexpression.

DROPBOX LINK

Please let me know if you have any questions or feedback!

Mike Howard
Product Manager, Oculus Avatars

Release notes

SDK

• Updated avatar meshes to include added eye / mouth geometry and blendshapes for expression and speech
• Updated avatar textures for skin, including mask information in the Green and Blue channels of the roughness texture
• Added runtime parameters to the avatar spec for eye color, lip color, brow color and lash color
• Added simulation models for eye gaze, mouth movement and general facial animation

Unity

• Updated shader to incorporate a mask channel, dynamically setting the eye / lip / lash / brow colors based on the user's spec
• Added support for alpha-to-coverage, removing the need to manage draw order for the avatar 'fade'
• Added support for alpha transparency on eyewear (enabling translucent eyewear)
• Added new linked library (OVRLipSync), which drives the audio → viseme processing for speech

Known issues

• We've added support across PC, Quest and Go. You'll need to be on the latest software update (which currently is in the process of rolling out) in order to use the Unity integration to build for Go and Quest, as there's a runtime library that needs to be updated. To make life a little more predictable, I've included the PC avatars .dll to load locally. This should enable you to get up and running until your android device software is updated to v. 3.58 or higher.
• We're still in the process of finalizing the package across all other versions of Unity and hardware, but it's been QA'd for 2018.2 against Rift, Go, Quest. The min spec will be 2017.4.11f1
• We've noted that in new Unity scenes, Vulkan support is enabled by default, which causes some issues with the APIs used by avatars. If you're seeing ghostly / translucent avatars, disable Auto Graphics API in the android player settings >other settings and remove everything except OpenGLES3 from the Graphics API's.
• You may receive a notice that
  OVRPlugin is out of date. Allow it to update to 1.32.0 and restart Unity.
• You may receive an API Update Required dialog (“This
  project contains scripts and/or assemblies that use obsolete APIs”) when
  importing the Oculus Integration. If so, click I Made a Backup, Go Ahead!

---

Implementation guide - Unity 2018.2

• Download the linked file file and unzip.
• Open Unity 2018.2 (Note that Unity 2018.3 currently is not recommended for any Oculus development) and create a new project.
• Go to the Unity Asset Store and import the Oculus Integration package (currently version 1.34). When importing, exclude the Avatars package by unchecking the box next to “Avatars” when the Import Unity Package window pops up.
  
• From the downloaded dropbox folder, import the unity package 'OvrAvatar.unityPackage'.
• Create a 'Plugins' folder under the project's 'Assets' folder. Copy the libovravatar.dll and OvrAvatarAssets.zip files into Assets/Plugins. This will override the default avatar runtime components in C:\Program Files\Oculus\Support\oculus-runtime
• Open the scene Oculus\Avatar\Samples\LocalAvatar\LocalAvatar.unity
• Set the app Id to be a valid app in 'Oculus→Avatars→Edit Settings→Oculus Rift App Id'
• Set the user ID to one of the test users' identified below.

User IDs

Please
conduct initial testing using one of the following user IDs to retrieve an Avatar
with an updated face mesh with expressive blends and no eyewear.

• 10150030458727564
• 10150030458738922
• 10150030458747067
• 10150030458756715
• 10150030458762178
• 10150030458769900
• 10150030458775732
• 10150030458785587
• 10150030458806683
• 10150030458820129
• 10150030458827644
• 10150030458843421

Going forward, all users will automatically be backfilled with an updated face mesh that takes advantage of both gaze and facial expressions. They'll have the option to remove their current eyewear option, exposing the eyes, as we launch this update into the avatar editors on PC and Go.

Setting Up Gaze Targets

As
an Avatar’s head moves around, different gaze targets enter and leave its field
of view, and its eyes automatically move between and focus on new gaze targets.

Any
object in a scene can be made a gaze target by doing the following:

1.    
Select any object in the scene.

2.    
Click the Tag
drop-down at the top of the Inspector.

3.    
Select Add
Tag.

4.    
Click the +
button under Tags to add a new tag.

5.    
Name the new tag GazeTarget and click Save.

Keep in mind the following considerations about gaze
targets:

• Avatar prefabs are already set to be gaze
  targets for other Avatars
• In the absence of specific gaze targets, the
  eyes will observe
  a gaze pattern with a distribution of points around directly forward
• You can toggle on/off the helper objects that
  show the current gaze targets being considered and the ...

[owenwp]

In case anyone else is still using an older version of Unity for their projects, this is working for me using 5.6.4p1-OC1, which was the special Oculus version of Unity 5 that had Dash support patched into it.

[MikeF]

@Ross_Beef
Is there any plan to add the ability to retrieve specific render parts if not using a combined mesh?

We have an ingame apparel system that requires certain components (mainly hair for now) to be disabled and currently our method of doing so will not work when players have the ability to disable eye wear. 

Using OvrAvatarMaterialManager.GetComponentType() works fine when the player has every slot equipped, but if they have no beard, hair or eye wear this will return incorrect information since that method relies on the game object's name which is set incrementally when a component is loaded in. 
ex: player only has body, clothing, hair. 
body_renderPart_0 (Body) ->returns correct 
body_renderPart_1 (Clothing) ->returns correct 
body_renderPart_2 (Hair) ->returns incorrectly as "eye wear" because of the 2 suffix assigned to the object.

[Ross_Beef]

Thanks @MikeF - raising this to our team and i’ll get back to you.