Skip to main content
Octonic
Inspiring
August 13, 2021
Solved

Hand Tracking Not Working With OpenXR Backend.

  • August 13, 2021
  • 12 replies
  • 9396 views

Hello, 

I have just upgraded to version 31.0 of the Oculus integration for unity 2020.3.0f1. It looks like hand tracking isn't working with the OpenXR backend. 
It works fine with the legacy plugin though. 

Does anyone else have the same issue? 

      This topic has been closed for replies.
      Best answer by korinVR

      I can confirm hands are not drawn. It seems hand tracking is not fully supported on the OpenXR backend for now. There is a related description in the release note.

       

      https://developer.oculus.com/downloads/package/unity-integration/

      Support for hand-tracking is currently restricted to the baseline OpenXR spec. Therefore, additional hand-tracking features such as collision capsules, hand input metadata, and runtime hand meshes are not yet supported. In addition, there is a known compatibility issue with the thumb trapezium bone (Thumb0) in the OpenXR-based OVRPlugin. We'll announce a fix when available in our release notes.

      12 replies

      korinVR
      korinVRAnswer
      Honored Contributor
      August 14, 2021

      I can confirm hands are not drawn. It seems hand tracking is not fully supported on the OpenXR backend for now. There is a related description in the release note.

       

      https://developer.oculus.com/downloads/package/unity-integration/

      Support for hand-tracking is currently restricted to the baseline OpenXR spec. Therefore, additional hand-tracking features such as collision capsules, hand input metadata, and runtime hand meshes are not yet supported. In addition, there is a known compatibility issue with the thumb trapezium bone (Thumb0) in the OpenXR-based OVRPlugin. We'll announce a fix when available in our release notes.
      Katsuomi Kobayashi ( @korinVR ) at FrameSynthesis Inc.
        prvncher
        New Participant
        August 15, 2021

        So I'm having the same issue with V31 (SDK + OS), alongside Unity 2019.4.29f1.

        Hands simply aren't detected and none of the hands sample scenes I've tried detect hands properly.
        (Train demo scene or Passtrough hands).

        When I switch back to the legacy backend, these demos properly detect hands, but then no longer support passthrough.

          drash
          Inspiring
          November 12, 2021

          With the v34 Unity integration, I confirmed that hand-tracking is still not working with the OpenXR backend.  So +1 to waiting on a fix for that!

          Titans of Space PLUS for Quest is now available on DrashVR.com
            DYosifov
            New Participant
            November 15, 2021

            By "it works fine with the legacy plugin" do you mean after building the APK or during the development with Oculus link? I cannot make it work with Oculus Link in Unity editor and the only way it works is by building and deploying to Quest (which slows the development time a lot).

            I am using v34 of Oculus integration, Unity 2020.3.22f1 and Oculus XR Plugin (tested both 1.10.0 and 1.11.0).

            OctonicAuthor
            Inspiring
            November 15, 2021

            No issue with legacy hand tracking in the editor on Oculus link in my 2020.3.20f1 project on v34. 

              TheDaddis
              Up and Coming
              November 27, 2021

              Still having the same issue with v34 in Unity 2020.3.22f1.

               

              Passthrough works if you use OpenXR, but hands are not visible.

               

              Hands work if you use Legacy VRAPI, but passthrough only appears black.

               

              Hopefully Oculus fixes hands on OpenXR, as it seems like that is the way forward: https://developer.oculus.com/blog/oculus-all-in-on-openxr-deprecates-proprietary-apis/

              cliv3dev
              cliv3dev
              New Participant
              December 1, 2021

              Basic hand tracking works with OpenXR backend and V34 and Unity 2020.3.x.

              But, as stated in the V34 Release note, there is a known bug on Thumb0 bone as you can see in this screenshot of my game Just Hoops:

               

              OpenXR V34 OVRplugin Thumb0 bug

               

              So if this doesn't bother you, you can use hand tracking anyway with OpenXR !

              I asked for an ETA on the bug fix to the support but no answer yet...

               

              Diego

              OctonicAuthor
              Inspiring
              December 2, 2021

              This is interesting. How did you manage to get the hand and bones to show up? 

              cliv3dev
              cliv3dev
              New Participant
              December 3, 2021

              I took the code from [...]\Assets\Oculus\VR\Scripts\Util\OVRSkeleton.cs and recreated my own data structure to handle all the bones updated by OVRPlugin and in this screenshot it's just a debug rendering of lines connected to each bone coming from the handtracking...

              OctonicAuthor
              Inspiring
              December 3, 2021

              Thanks for the reply. How did you get the bones from OVRPlugin? It looks like getskeleton2 is returning false and nothing is being initialized in OVRSkeleton. 

              cliv3dev
              cliv3dev
              New Participant
              December 3, 2021

              First I check handtracking status of each hand with GetHandState(), and when it returns true, I query a skeleton with GetSkeleton2()...

              See the code:

               

               
                      OVRPlugin.HandState _handState = new OVRPlugin.HandState();
                      //bool queryHandState(Feature f)
                      {
                          bool isTracked=false;
                          if (OVRPlugin.GetHandState(OVRPlugin.Step.Render, f==Feature.HandTrackingLeft?OVRPlugin.Hand.HandLeft:OVRPlugin.Hand.HandRight, ref _handState))
                              isTracked = (_handState.Status & OVRPlugin.HandStatus.HandTracked) != 0;
                          if (isTracked)
              {
               if (OVRPlugin.GetSkeleton2(_featureType==Feature.HandTrackingLeft?OVRPlugin.SkeletonType.HandLeft:OVRPlugin.SkeletonType.HandRight, ref _skeleton)) { initBones(); ...}
              }
              Terms & ConditionsAccessibility statement