Meta Avatars: how to feed custom input?

[Anonymous]

Hi,

I'm prototyping with the new Meta Avatars system and am using the SampleAvatarEntity script to instantiate fake avatars. I'd like to replay head pose, controller states, and/or hand states (positions) in order to drive these avatars from pre-recorded data in my scene.

Using SampleInputManager (included with the avatar examples) as a starting point, I assumed all I had to do was provide my own InputTrackingDelegate, InputControlDelegate, and (for hand tracking), HandTrackingDelegate. I started with this, feeding in fake controller inputs and marking the controllers as active but it has absolutely no effect. The avatar spawns in t-pose and does nothing.

public class ReplayAvatarInputManager : OvrAvatarInputManager
{
    protected void Awake()
    {
    }

    private void Start()
    {
       if (BodyTracking != null)
       {
            BodyTracking.InputTrackingDelegate = new ReplayInputTrackingDelegate();
            BodyTracking.InputControlDelegate = new SampleInputControlDelegate();
        }
    }

    protected override void OnDestroyCalled()
    {
        base.OnDestroyCalled();
    }
}

public class ReplayInputTrackingDelegate : OvrAvatarInputTrackingDelegate
{
    public ReplayInputTrackingDelegate()
    {
    }

    public override bool GetRawInputTrackingState(out OvrAvatarInputTrackingState inputTrackingState)
    {
        inputTrackingState = new OvrAvatarInputTrackingState();
        inputTrackingState.headsetActive = true;
        inputTrackingState.leftControllerActive = true;
        inputTrackingState.rightControllerActive = true;
        inputTrackingState.leftControllerVisible = true;
        inputTrackingState.rightControllerVisible = true;
        inputTrackingState.headset.position = Vector3.zero;
        inputTrackingState.headset.orientation = Quaternion.identity;
        inputTrackingState.headset.scale = Vector3.one;
        inputTrackingState.leftController.position = 2*Vector3.one;
        inputTrackingState.rightController.position = Vector3.one;
        inputTrackingState.leftController.orientation = Quaternion.identity;
        inputTrackingState.rightController.orientation = Quaternion.identity;
        inputTrackingState.leftController.scale = Vector3.one;
        inputTrackingState.rightController.scale = Vector3.one;
        return true;
    }
}

The delegate method is called, I've confirmed, but the data it returns is not being used. What am I doing wrong? I've made sure to connect this ReplayAvatarInputManager behavior to the Tracking Input property on SampleAvatarEntity.

Thanks,

Bart

[j7-Hitstream]

Thanks for the post - this sounds like it should be something part of the AVATAR2 kit -
Were do you use this script?
- I've got a basic XR Rig - and I've added the left/right/head points and added the script to the AvatarSDKManagerHorizon prefab, also disabled the SampleInputManager, but the avatar still stays in the T-pose, with no movements. 
Any ideas?

[Viconthebeach]

Great, thank you for your code sample, I was very close to that but I missed the GetTrackingSpacePose and nothing about it found in the guides.

However, the pose is only applied when not using another avatar as 1st person with real hands tracking. Do you know why the OVRCameraRig's tracking seems to override the custom hand pose tracking?

[Viconthebeach]

I am trying to get the same result, a simple avatar static pose, but even using your code (who seems to work for you) I still get the CameraRig tracking overriding my custom pose. Everythings fine in Play mode but when running on headset it still takes input from the CameraRig, ignoring the custom trackingState.

I'm going mad with it for a couple of days now, I also did try using the FilterInput trick explained here without success https://developer.oculus.com/documentation/unity/meta-avatars-custom-input-handling/

Do you have an idea on what 's possibly going wrong?

[Ludwan93]

Did you manage to find a workaround for this?

I'm having the exact same issue. Whenever I have hand-tracking enabled it overrides the custom pose. 

[simonmersus]

To add on to the discussion (and to reply to the two last comments), this code used to work for me. I was working with Oculus SDK version 54-55 in Unity 2021 and the custom avatar input script did the trick for a few months, but recently upgraded my project to Unity 2022 + Meta SDKs v60-62 and now it behaves like you say. The hand poses are still working, but the Avatar is not following the wrists positions anymore but my own hands (with Standalone selected). If I hide my hands, it just stops moving. And if I select None instead of Standalone, I get a T-pose. So yeah they definitely changed something in the last 6 months.