cancel
Showing results for 
Search instead for 
Did you mean: 

Problem with Oculus Avatars replication in ue4 multiplayer

OrionPS
Honored Guest
Hello Everyone,

I'm trying to make multiplayer in Unreal Engine 4.15 and to implement Oculus Avatars as an representation of players.
Problem I'm having is that the client doesn't see avatars at all and movement of all avatars is copied from avatar possessed by server (server sees all avatars).  
I have managed to get personalized avatars from oculus platform for server and client player characters but still movement is copied from server and clients can not see avatars at all.

Hierarchy looks like this:  
I have player character blueprint that manages player movement and all interactions.  
This Player Character has child actor added as component of localAvatar class which is copied from AvatarSample project shared by Oculus.

Here is how I have set replication for child actor component:
d0c6vmjq9k1d.png

I tried to set different replication methods for event that initializes avatars but I couldn't get the desired effect.  
When I changed replication method of initializePlayerAvatar to replicated on client, client was able to see it's own and server's avatars but server could see client avatar.  

This event looks like this and it's called by pressing U key:
r033mdfqmbqi.png

InitializeAvatar method:


    void ALocalAvatar::InitializeAvatar(FString OculusId)
    {
    IOnlineIdentityPtr IdentityInterface = Online::GetIdentityInterface("OculusSubsystemPlatform");
    if (IdentityInterface.IsValid())
    {
    OnLoginCompleteDelegateHandle = IdentityInterface->AddOnLoginCompleteDelegate_Handle(0, FOnLoginCompleteDelegate::CreateUObject(this, &ALocalAvatar::OnLoginComplete));
    IdentityInterface->AutoLogin(0);
    }
    uint64 OculusIdAsUInt64 = FCString::Strtoui64(*OculusId, NULL, 10);
    if (AvatarComponent)
    {
    AvatarComponent->RequestAvatar(OculusIdAsUInt64);
    }
    
    AvatarHands[ovrHand_Left] = nullptr;
    AvatarHands[ovrHand_Right] = nullptr;
    }

So my questions are:  
How to separate avatar movement from server so that movement is not copied between all avatars?  
How to properly replicate avatars so they could be seen both by server and clients?

Best regards,  
OrionPS
7 REPLIES 7

pieterdub
Explorer
Hi OrionPS,
I have been working on a multiplayer Avatar sample in UE.  In my sample, I am spawning the avatar pawns on the server with the following replication settings:
bReplicates = true; 
bAlwaysRelevant = true;
bReplicateMovement = true;

Client doesn't see Avatars at all:
Where are you spawning the Avatar?  It it part of the scene?  Or are you spawning it in BP?
Does the client see the locally controlled Avatar?

I believe you need to spawn the Avatars on the server so replication copies them to all clients.
In my GameMode (only runs on server), I use the OnPostLogin event to spawn each player's avatar and possess that pawn with the player controller that is a parameter of the OnPostLogin event:
ap4eqxj0oia5.png

here is the spawn that is down the line from the OnPostLogin:
jtst12olj2gf.png

So once the avatar's have been spawned and possessed with player controllers on the server, they are automatically replicated to all clients.

Movement:
When you say movement of the avatars is copied from the Server avatar, do you mean the movement of the hands/head etc or are you moving the avatar around with the sticks (moving the position of the avatar)?  I assume you mean the hand/head movement but let me know if not.

In my sample, I am recording avatar movement packets for the local avatar, and sending them to the server who then replicates those packets out to all clients, and then the clients play them back.

Did you make any other modifications to the LocalAvatar code or are you using it as it was provided in the sample?  I think the code in the sample is recording the movement data packets from the local avatar into a set of queues and that is playing back movement to all avatars (see void FOvrAvatarManager::QueueAvatarPacket(ovrAvatarPacket* packet) ).  Right now, that function is writing the packet into all the queues in AvatarPacketQueues and isn't keeping track of which avatar the packet belongs to.  So the server writes data into this queue, and then it is replicated to all clients which isn't correct.  It was setup this way because the sample is single player and allows the user to spawn multiple remote avatars that all mimic the movement of the local avatar.

What you would need to do, is set the Key of the AvatarPacketQueue and use that to keep the queues separate.  You could used the player's UserID as the Key for each queue.  I believe you would also need to make an RPC call from each client to send the packet data to the server and then the server writes that into the correct queue in the AvatarManager.  

I don't currently use the AvatarManager in my sample.  I send the movement packets from the clients to the server, and each Avatar Pawn maintains a queue.  Server puts the correct packets into each avatar's queue using a PacketKey, and then replication copies that to each client machine.  Then in the tick function for any nonlocal avatars, I playback movement from its queue.

Let me know if this helps at all.  Hopefully I will have the sample ready to distribute within a couple weeks.

m_milazzo
Protege

pieterdub said:

Hi OrionPS,
I have been working on a multiplayer Avatar sample in UE.  In my sample, I am spawning the avatar pawns on the server with the following replication settings:
bReplicates = true; 
bAlwaysRelevant = true;
bReplicateMovement = true;

Client doesn't see Avatars at all:
Where are you spawning the Avatar?  It it part of the scene?  Or are you spawning it in BP?
Does the client see the locally controlled Avatar?

I believe you need to spawn the Avatars on the server so replication copies them to all clients.
In my GameMode (only runs on server), I use the OnPostLogin event to spawn each player's avatar and possess that pawn with the player controller that is a parameter of the OnPostLogin event:
ap4eqxj0oia5.png

here is the spawn that is down the line from the OnPostLogin:
jtst12olj2gf.png

So once the avatar's have been spawned and possessed with player controllers on the server, they are automatically replicated to all clients.

Movement:
When you say movement of the avatars is copied from the Server avatar, do you mean the movement of the hands/head etc or are you moving the avatar around with the sticks (moving the position of the avatar)?  I assume you mean the hand/head movement but let me know if not.

In my sample, I am recording avatar movement packets for the local avatar, and sending them to the server who then replicates those packets out to all clients, and then the clients play them back.

Did you make any other modifications to the LocalAvatar code or are you using it as it was provided in the sample?  I think the code in the sample is recording the movement data packets from the local avatar into a set of queues and that is playing back movement to all avatars (see void FOvrAvatarManager::QueueAvatarPacket(ovrAvatarPacket* packet) ).  Right now, that function is writing the packet into all the queues in AvatarPacketQueues and isn't keeping track of which avatar the packet belongs to.  So the server writes data into this queue, and then it is replicated to all clients which isn't correct.  It was setup this way because the sample is single player and allows the user to spawn multiple remote avatars that all mimic the movement of the local avatar.

What you would need to do, is set the Key of the AvatarPacketQueue and use that to keep the queues separate.  You could used the player's UserID as the Key for each queue.  I believe you would also need to make an RPC call from each client to send the packet data to the server and then the server writes that into the correct queue in the AvatarManager.  

I don't currently use the AvatarManager in my sample.  I send the movement packets from the clients to the server, and each Avatar Pawn maintains a queue.  Server puts the correct packets into each avatar's queue using a PacketKey, and then replication copies that to each client machine.  Then in the tick function for any nonlocal avatars, I playback movement from its queue.

Let me know if this helps at all.  Hopefully I will have the sample ready to distribute within a couple weeks.


How did you replicate ovrAvatarPacket? (since it is neither a UCLASS, nor a USTRUCT nor a UENUM)

Is it possible to have this sample? It would be very useful for a lot of users. Thanks!

pieterdub
Explorer
Sample is super close to being released!  Apologies for the delay!  

WRT the data replication, you are correct that you can't just replicate the AvatarPacket.  I take the data in the packet and copy it into a USTRUCT to be sent over the wire.

In my NetworkAvatar.h:

class UOvrAvatar:

USTRUCT()
struct FAvatarPacket2
{
  GENERATED_USTRUCT_BODY()

  UPROPERTY()
  TArray<uint8> AvatarPacketData;
  UPROPERTY()
  uint32 packetSequenceNumber;
};
UCLASS()
class AVATARSAMPLES_API ANetworkAvatar : public APawn
{
  GENERATED_BODY()
  private:
    ovrAvatarPacket* CurrentPacket = nullptr;
    FAvatarPacket2 CurrentPacketStruct;
...

Then in NetworkAvatar.cpp in my UpdatePacketRecording function which is runs only on the locally controlled avatar instance:
if (PacketSettings.AccumulatedTime >= PacketSettings.UpdateRate)
{
  PacketSettings.AccumulatedTime = 0.f; //reset the recording timer
  ovrAvatarPacket* packet = AvatarComponent->EndPacketRecording();
  if (packet == nullptr)
  {
    //nothing to do, so start recording again
    UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - packet == nullptr"));
    AvatarComponent->StartPacketRecording();
    return;
  }
  UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - created a packet"));
  uint8_t* Buffer;
  uint32_t BufferSize;

  BufferSize = ovrAvatarPacket_GetSize(packet);
  UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - BufferSize =   ovrAvatarPacket_GetSize(): %d"), BufferSize);
  Buffer = new uint8_t[BufferSize];
  if (ovrAvatarPacket_Write(packet, BufferSize, Buffer)) 
  {
    UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - packing USTRUCT"), BufferSize);
    //pack data into our Struct array
    CurrentPacketStruct.AvatarPacketData.Append(Buffer, BufferSize);
    ServerHandleAvatarPacket(CurrentPacketStruct); //RPC Call to server with USTRUCT data.
...

and then on the opposite side, when you are receiving a packet from the server and need to play it back:
.h: 
UPROPERTY(ReplicatedUsing = OnRep_ReceivedPacket)
  FAvatarPacket2 R_AvatarPacket;

cpp:
void ANetworkAvatar::OnRep_ReceivedPacket()
{
  //Check if the incoming packet belongs to a remote avatar rather than the local (we don't want to move the local as that    is user controlled)
  if (!bAvatarIsLocal)
  {
    UE_LOG_ONLINE(Verbose, TEXT("ANetworkAvatar::OnRep_ReceivedPacket() for non-local avatar: %s with   SequenceNum: %d"), *PacketKey, R_AvatarPacket.packetSequenceNumber);

  uint32_t BufferSize;
  uint8_t* Buffer;
  BufferSize = R_AvatarPacket.AvatarPacketData.Num();
  Buffer = new uint8_t[BufferSize];
  for (uint32_t elementIdx = 0; elementIdx < BufferSize; elementIdx++)
  {
    Buffer[elementIdx] = R_AvatarPacket.AvatarPacketData[elementIdx];
  }
  FOvrAvatarManager::Get().QueueAvatarPacket(Buffer, BufferSize, PacketKey,   R_AvatarPacket.packetSequenceNumber);

  delete[] Buffer;
  }
}

Neontop
Heroic Explorer
Hello @pieterdub I'm having the same problem with the AvatatSamples from SDK 1.27.
Is is possible to have access to those codes.
Thank you



beaulima9933
Expert Protege
If anyone wants access to the code for UE4.19 - 4.20 multiplayer just PM me :smile:

Neontop
Heroic Explorer
@beaulima9933 thanks for the link.
I'm trying to integrate the platform SDK so it will be more easy to access Oculus Online Subsytem functionalities.

Neontop
Heroic Explorer
Anyone tried to implement the code of @beaulima9933 here ?