[solved] --IMPORTANT-- UNITY avatar remote loopback not working - need feedback from avatar team

[Yoirgl]

hey guys.

Desperate by the removal of the custom pose system with 1.14  and 1.15, we decided to go have a look at the remote loopback as you guys created it.
But even on the demo scene it's quite bad.
it has a delay of 0.5 seconds at start and if you turn off packet recording and back on, synchronisation seems totally off.
Would it be possible to please put back the custom controls on our character's pose of 1.13 and removing the "black box" of IntPtr ovrNativePacket which doesn't make any sense ?
Why denying us access to poseframes and custom updates of the avatar ?
Why changing  OvrAvatarPacket class ?
Please do something about it. We would really like to use oculus avatar system but clearly this is not working and from what we've been reading we're far from being the only ones having both those issues (consider contacting MikeF about it too)
Can we expect a more customizable avatar pose transfer in the future or do you plan on keeping it the way it is ? It's important for us to have that information because if avatar start being unusable for us we need to spend a lot of time coding an alternative.

Please send us a bit more information.

Yoirgl.

[MikeF]

2nd'd! the old system was much better to work with for every reason Yoirgl mentioned

[Wilzonz]

Hi @Yoirgl  and @MikeF , 

Could you confirm that you are using the latest version? There was a bug that shipped with the first update which has been fixed, that fits the poor behavior you are seeing. (the default update rate was way too slow, and there was a bug that caused the RemoteAvatar to get back logged on updates and never catch up to the latest). 

In OvrAvatar.cs could you confirm this structure:

[System.Serializable]

public class PacketRecordSettings

{

    internal bool RecordingFrames = false;

    public float UpdateRate = 1f / 30f; // 30 hz update of packets

    internal float AccumulatedTime;

};

Please note that the UpdateRate is meant to be tuned to suit your networking needs for your application, and should be editable in the Editor on the Avatar's properties. 

The ovrNativePacket is an optimized structure that the SDK uses internally. It compresses the pose data, and also sends a minimal set of transform updates, so should give you much better network performance than the Unity side pose blending which was in the first version.  

The RemoteLoopback sample has been working well for us to demo/test this flow, but it's possible you are seeing a bug we did not discover.   We also have an unreleased demo app using this code successfully, so I'd like to work with you to help sort out your issues.

Sorry for the troubles,

Wilzonz

[Yoirgl]

Hello.


@Wilzonz Thanks for your reply!

Wilzonz said:

Hi @Yoirgl  and @MikeF , 

Could you confirm that you are using the latest version? There was a bug that shipped with the first update which has been fixed, that fits the poor behavior you are seeing. (the default update rate was way too slow, and there was a bug that caused the RemoteAvatar to get back logged on updates and never catch up to the latest). 

In OvrAvatar.cs could you confirm this structure:

[System.Serializable]

public class PacketRecordSettings

{

    internal bool RecordingFrames = false;

    public float UpdateRate = 1f / 30f; // 30 hz update of packets

    internal float AccumulatedTime;

};

For me after downloading 1.15 and updating my project i still have

 public bool RecordingFrames = false;
    public float UpdateRate = 1.0f / 5.0f;  // Lower rate = bigger packets. Try to optimize against fidelity vs Network Overhead.
    public float AccumulatedTime = 0.0f;

So clearly for me it didn't get updated.

The ovrNativePacket is an optimized structure that the SDK uses
internally. It compresses the pose data, and also sends a minimal set of
transform updates, so should give you much better network performance
than the Unity side pose blending which was in the first version. 

I understand however I was able to compress much more than what you guys do as for some elements i didn't need so much precision (rotations narrowed to 3 bytes (256 values for each 360°) position floats narrowed to shorts due to centimetre precision etc...) Isn't it possible to keep both systems coexisting ?

I'll try to create a new blank project and  test again if I face the same latency issue there.
I'll let you know of my findings.

Thanks.

Yoirgl.

[chrisnolet]

Interesting... I have an older version 1.15, too – running at 5 Hz – and I noticed the increased lag. Is the newer 1.15 available at https://developer.oculus.com/downloads/package/oculus-avatar-sdk? Are there any other changes, or is it just that the UpdateRate has changed?

I don't have any strong opinions on the 'black box' IntPtr to ovrNativePacket. I was concerned that quaternions were being sent uncompressed in the old version, and it sounds like 1.14+ uses at least 'smallest three' and maybe some other tricks, so I think that's a win. It would be good to keep those savings regardless of whether or not the C# wrapper is re-introduced.

[Wilzonz]

Stay tuned folks, it appears the bug fixes may have gotten lost in the update.  If you are still seeing that old code, then you will be seeing the accumulated lag that was fixed with the latest. 

Will update here soon,

Wilzonz

[chrisnolet]

Understood. Just to answer my own question and clarify things: the 'new' 1.15 isn't available on developer.oculus.com yet. I just downloaded the latest and the code was the same. (Hopefully we can expect a version 1.15.1 soon.)

[Yoirgl]

Wilzonz said:

Stay tuned folks, it appears the bug fixes may have gotten lost in the update.  If you are still seeing that old code, then you will be seeing the accumulated lag that was fixed with the latest. 

Will update here soon,

Wilzonz

thanks for checking it out 🙂

Can you tell us if something will be changed in the future for customization of packets sent as an alternative to the automatic system  ?

Yoirgl.

[Wilzonz]

Hey folks, 

We released an update to 1.15, which has bug fixes for the new packet updates.  The bug where the Remote Avatar would slowly drift into the past on updates and never catch up is fixed, and the packet update rate is exposed to the Editor, on the Avatar.  You can tune the update to as frequent as you'd like for your use case.

@Yoirgl - we discussed your concerns, and will aim to have the old pose blending in Unity available as an alternative to the SDK optimized flow for the next release.  You are correct that you are squishing things a bit tighter than our SDK settings, so enabling you to do so seems like a great use case for this code.

Sorry for the hiccups all, things should be working better now!

Wilzonz

[Yoirgl]

Wilzonz said:

we discussed your concerns, and will aim to have the old pose blending in Unity available as an alternative to the SDK optimized flow for the next release.

Huzza !

Thanks for that. I'm happy you guys took our concerns into consideration. 🙂

Yoirgl.