Skip to main content
AlienCurv
Inspiring
April 24, 2022
Solved

Getting head and finger orientation

Hey gang - glad to see a great support forum for the Oculus Quest 2 with Unity. I'm migrating an app from open XR to oculus to take advantage of the oculus hand tracking. I'm experiencing issues with the data (or lack of) and wondering if anyone can provide advice.

 

I'm using our existing client/server framework to send data to the server. The data is being sent and received correctly - but the data is strange.

 

The finger data never changes from the skeleton. During testing, I'm only sending the tip of the index finger on the right hand. The outcome is to get the amount of bend of each finger. 

 

The camera transform sends the correct data, and it's working great. We just can't figure out how to get the finger data to know the position of each finger.

 

 

void Update() {

        // send the index finger data
        OVRPlugin.Skeleton skeleton;
        if (OVRPlugin.GetSkeleton(OVRPlugin.SkeletonType.HandRight, out skeleton))
          foreach (var bone in skeleton.Bones)
            if (bone.Id == OVRPlugin.BoneId.Hand_Index3)
              _streamClient.SendPacket(
                StreamClient.CommandEnums.FingerRight0,
                MapToByte(bone.Pose.Orientation.x),
                MapToByte(bone.Pose.Orientation.y),
                MapToByte(bone.Pose.Orientation.z));

        // send head data (pitch, yaw, roll)
        _streamClient.SendPacket(
          StreamClient.CommandEnums.pitchYawRoll,
          MapToByte(Camera.current.transform.localRotation.x),
          MapToByte(Camera.current.transform.localRotation.y),
          MapToByte(Camera.current.transform.localRotation.z));
}

 

 

 

 

Any assistance is greatly appreciated - thanks in advance!

AlienCurvAnswer
Inspiring
April 24, 2022

Oh, I think I discovered the bone rotation index. I'll write this down here so anyone in the future can find it. There's a documented list for the skeleton, which partially applies to the bone rotation array. I'll paste it here because it's confusing when the bone rotation and skeleton data are a little different per this page: Set Up Hand Tracking | Oculus Developers

 

The bone rotation array contains the following...

 

Hand_WristRoot = 0 // root frame of the hand, where the wrist is located
Hand_ForearmStub = 1 // frame for user's forearm
Hand_Thumb0 = 2 // thumb trapezium bone
Hand_Thumb1 = 3 // thumb metacarpal bone
Hand_Thumb2 = 4 // thumb proximal phalange bone
Hand_Thumb3 = 5 // thumb distal phalange bone
Hand_Index1 = 6 // index proximal phalange bone
Hand_Index2 = 7 // index intermediate phalange bone
Hand_Index3 = 8 // index distal phalange bone
Hand_Middle1 = 9 // middle proximal phalange bone
Hand_Middle2 = 10 // middle intermediate phalange bone
Hand_Middle3 = 11 // middle distal phalange bone
Hand_Ring1 = 12 // ring proximal phalange bone
Hand_Ring2 = 13 // ring intermediate phalange bone
Hand_Ring3 = 14 // ring distal phalange bone
Hand_Pinky0 = 15 // pinky metacarpal bone
Hand_Pinky1 = 16 // pinky proximal phalange bone
Hand_Pinky2 = 17 // pinky intermediate phalange bone
Hand_Pinky3 = 18 // pinky distal phalange bone

 

View in thread

8 replies

AlienCurvAuthor
Inspiring
April 24, 2022

After posting this, I found a different method that returns me the hand state. I can see movement values, but I'm not sure what index is in the array for each finger of BoneRotations[].  Looking at the OVR Plugin code, I see a lot of BoneRotations, but how can I know which index is for what bone? 

 

I "think" index 7 is for the index finger? Is there any additional information about the bones for the skeleton that I can reference?

 

        OVRPlugin.HandState handState = default(OVRPlugin.HandState);

        if (OVRPlugin.GetHandState(OVRPlugin.Step.Render, OVRPlugin.Hand.HandRight, ref handState)) {

          if (handState.BoneRotations.Length > 9)
            _streamClient.SendPacket(
              StreamClient.CommandEnums.FingerRight0,
              MapToByte(handState.BoneRotations[7].x),
              MapToByte(handState.BoneRotations[7].y),
              MapToByte(handState.BoneRotations[7].z));
        }

 

AlienCurvAuthorAnswer
Inspiring
April 24, 2022

Oh, I think I discovered the bone rotation index. I'll write this down here so anyone in the future can find it. There's a documented list for the skeleton, which partially applies to the bone rotation array. I'll paste it here because it's confusing when the bone rotation and skeleton data are a little different per this page: Set Up Hand Tracking | Oculus Developers

 

The bone rotation array contains the following...

 

Hand_WristRoot = 0 // root frame of the hand, where the wrist is located
Hand_ForearmStub = 1 // frame for user's forearm
Hand_Thumb0 = 2 // thumb trapezium bone
Hand_Thumb1 = 3 // thumb metacarpal bone
Hand_Thumb2 = 4 // thumb proximal phalange bone
Hand_Thumb3 = 5 // thumb distal phalange bone
Hand_Index1 = 6 // index proximal phalange bone
Hand_Index2 = 7 // index intermediate phalange bone
Hand_Index3 = 8 // index distal phalange bone
Hand_Middle1 = 9 // middle proximal phalange bone
Hand_Middle2 = 10 // middle intermediate phalange bone
Hand_Middle3 = 11 // middle distal phalange bone
Hand_Ring1 = 12 // ring proximal phalange bone
Hand_Ring2 = 13 // ring intermediate phalange bone
Hand_Ring3 = 14 // ring distal phalange bone
Hand_Pinky0 = 15 // pinky metacarpal bone
Hand_Pinky1 = 16 // pinky proximal phalange bone
Hand_Pinky2 = 17 // pinky intermediate phalange bone
Hand_Pinky3 = 18 // pinky distal phalange bone

 

    hasenav
    Participant
    April 25, 2022

    Hey, I'm trying to calculate the angle between the hand and the wrist but I can't find it. I have tried to calculate the angle between Hand_WristRoot and Hand_ForearmSt but it does not work for me. Thanks.

    DaeK
    Participant
    August 25, 2025

    This works!!! Thank you so much for putting your solution. However, I was not able to find documentation about OVRPlugin.GetHandState... How did you learn about this?

      Terms & ConditionsAccessibility statement