Skip to main content
BattleAxeVR
Inspiring
October 2, 2023
Solved

XR_METAX1_simultaneous_hands_controllers_management OpenXR not working

  • October 2, 2023
  • 14 replies
  • 7271 views

I thought I created a post about this already but I can't see it anywhere.

 

Anyway, I've been trying to run the new XrHandsAndControllers OpenXR SDK sample in v56 SDK on my Quest Pro, and build.bat won't even compile.

I get this error:

 

 

Aside from that, XR_METAX1_simultaneous_hands_controllers_management OpenXR extension isn't listed on my Quest Pro with v57 firmware and v56 SDK (which is where I got the headers for this from).

 

I added the hand tracking permission and experimental flags in the manifest, just like XrSample above has, but it doesn't enumerate the XR_METAX1_simultaneous_hands_controllers_management OpenXR extension by the runtime.

Can anyone help me with either or both of these issues? Thanks.

 

 

Best answer by CountToSeven

Would you kindly try using Android Studio versions prior to 2020.3.1 along with Java 1.8 or 11? For the NDK, all versions should be compatible except the most recent one. As for the SDK, versions up to and including 30 have been proven to work. If you encounter any issues, you might also consider testing with Android SDK 25 / 26. We've verified internally that these configurations should be functional.

I recognize that our current SDK may seem outdated. As johnkearney  has pointed out, we're actively working on updating the Oculus OpenXR Mobile SDK. Thank you so much for your continued interest and support for Meta's products.

14 replies

BattleAxeVRAuthor
Inspiring
October 2, 2023

johnkearney any ideas? If I could only run the Xr Sample I could debug my app and get this working, but alas the samples won't even build.

johnkearney
Meta Employee
October 3, 2023

Hi BattleAxeVR - I seem to be having problems with the forum today and my posts are getting deleted. Trying a smaller post to see if that gets deleted too before I write the same thing again!

johnkearney
Meta Employee
October 3, 2023

Hi BattleAxeVR

1. I'm happy to hear that you have a Quest Pro again!
2. XR_METAX1_simultaneous_hands_controllers_management is an experimental API and we would love any feedback you have on the API.
3. To enable experimental APIs you have to do two things: Add an AndroidManifest entry; and calling an adb shell command. Docs: (https://developer.oculus.com/documentation/native/android/mobile-experimental-features/)
4. As CountToSeven  says above, we have seen issues building the samples with newer NDKs and will release a version of the samples which builds with newer NDKs soon but this will likely cause problems for older NDKs so we are going to be careful and slow as we roll this out.

Thanks,

John

CountToSeven
New Participant
October 2, 2023

Do you have a list of spec including NDK version, SDK version, and Java version. It is possible that your Java version or NDK version are too new which caused this issue. 

BattleAxeVRAuthor
Inspiring
October 3, 2023

I'm sure you're right, but I just don't have time/patience to install a bunch of random permutations of those three in my machine and wreck my ability to code on my main Android Studio apps.

 

I have NDKs 20.0 all the way to 25.2 installed, SDK is only latest, 11.0, and my Java version is :

 

java --version

openjdk 17.0.6

 

It may be simpler if you know, what versions of these DO work, and I'll see about installing those on another machine and build these sample APKs from there. I appreciate your help btw.

CountToSevenAnswer
New Participant
October 3, 2023

Would you kindly try using Android Studio versions prior to 2020.3.1 along with Java 1.8 or 11? For the NDK, all versions should be compatible except the most recent one. As for the SDK, versions up to and including 30 have been proven to work. If you encounter any issues, you might also consider testing with Android SDK 25 / 26. We've verified internally that these configurations should be functional.

I recognize that our current SDK may seem outdated. As johnkearney  has pointed out, we're actively working on updating the Oculus OpenXR Mobile SDK. Thank you so much for your continued interest and support for Meta's products.

    BattleAxeVRAuthor
    Inspiring
    October 3, 2023

    Thanks John, with that adb command the extension shows up in the enumated XR extensions so I guess my problem compiling the samples is moot. I'll let you know how it works in my RPG where I have 2-handed weapons (and use BeatSaber grip extensions for Touch Pros so I can grip the bottom half with my offhand), then be able to use my offhand freely to manipulate objects or cast spells or draw arrows, etc. All I need is 1 thumbstick and set of buttons, offhand via hand tracking can hold a shield just fine without a controller.

    Just a README with the prerequisites you need for those build.bat files to run properly would be OK.

    Yeah I got another Quest Pro for free from a client, so I'm using it as my daily driver for my PC VR path tracing game using eye-tracking and body-tracking for waist-oriented locomotion  (which completely solved VR sim sickness)

    I know I've asked you this before, but I would really, really appreciate if you could get around to making all these OpenXR features work on Link/AirLink without a dev account, so our customers can use them without having to jump through extra hoops: hand / body / face / eye tracking, passthrough. Everything, basically.

    If you can make it work on a Standalone there should be feature parity on PC ideally, and not just to reduce brutal Android iteration times, but to increase sales of games for games that support both Standalone and PC.

    Even if not right away, at least eventually. I don't understand why even hand tracking is still considered "experimental" or "developer only" over Link. It seems overly restrictive and something Apple would do. Please be better than Apple!

    BattleAxeVRAuthor
    Inspiring
    October 3, 2023

    Yes! It worked on the first try, I will use this extensively in my game if you'll allow it over Link.

    Please consider it!

    https://twitter.com/BattleAxeVR/status/1709068926200566221

      BattleAxeVRAuthor
      Inspiring
      October 3, 2023

      I own all Oculus / Meta headsets except Rift CV1, and that won't change with Quest 3! I look forward to getting mine on day 1. Thanks again. I appreciate all the help and am excited to see how well AirLink looks / runs on Quest 3. But more important than that, is just getting rid of the "developer account" requirement. It blocks Unity shipping builds (for those who use it, which is a lot), from using those features at all on PC builds, which actively harms us. I have many projects cooking for myself and for my clients on Quest standalone mode, but the whole promise of OpenXR is that the software should be portable, so having a Quest standalone game have the same features available on a shipping PC game or app would be a huge improvement.

      BattleAxeVRAuthor
      Inspiring
      October 3, 2023

      ..

      BattleAxeVRAuthor
      Inspiring
      October 4, 2023

      While attempting to port this to my hello_xr demo branch, 

      Oculus OpenXR runtime status report:
       
      Public test channel with developer runtime features enabled, Oculus App Version 57.0.0.254.153 (57.0.0.254.153)
       
      [23:05:43.152][Info   ]  Available OPENXR Extensions: (56)
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_D3D11_enable SpecVersion=9
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_D3D12_enable SpecVersion=9
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_opengl_enable SpecVersion=10
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_vulkan_enable SpecVersion=8
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_vulkan_enable2 SpecVersion=2
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_composition_layer_depth SpecVersion=6
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_win32_convert_performance_counter_time SpecVersion=1
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_convert_timespec_time SpecVersion=1
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_composition_layer_cube SpecVersion=8
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_composition_layer_cylinder SpecVersion=4
      [23:05:43.152][Info   ] OPENXR Extension:   Name=XR_KHR_composition_layer_equirect SpecVersion=3
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_KHR_visibility_mask SpecVersion=2
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_KHR_composition_layer_color_scale_bias SpecVersion=5
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_EXT_win32_appcontainer_compatible SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_EXT_debug_utils SpecVersion=5
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_OCULUS_recenter_event SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_OCULUS_audio_device_guid SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_FB_color_space SpecVersion=3
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_FB_display_refresh_rate SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_META_performance_metrics SpecVersion=2
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_META_headset_id SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_OCULUS_ovrsession_handle SpecVersion=1
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_EXT_hand_tracking SpecVersion=4
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_FB_hand_tracking_aim SpecVersion=2
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_FB_hand_tracking_capsules SpecVersion=3
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_FB_hand_tracking_mesh SpecVersion=3
      [23:05:43.153][Info   ] OPENXR Extension:   Name=XR_METAX1_simultaneous_hands_controllers_management SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_body_tracking SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_eye_tracking_social SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_face_tracking SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_keyboard_tracking SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_passthrough SpecVersion=3
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_triangle_mesh SpecVersion=2
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_META_passthrough_color_lut SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_METAX1_passthrough_color_lut SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_META_passthrough_preferences SpecVersion=1
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_render_model SpecVersion=3
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_container SpecVersion=2
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_scene SpecVersion=4
      [23:05:43.154][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity SpecVersion=2
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_storage SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_query SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_user SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_storage_batch SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_spatial_entity_sharing SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_META_spatial_entity_mesh SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_META_virtual_keyboard SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_haptic_amplitude_envelope SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_haptic_pcm SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_touch_controller_pro SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_META_touch_controller_plus SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FB_touch_controller_proximity SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_OCULUS_external_camera SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_FBX1_touch_controller_extras SpecVersion=1
      [23:05:43.155][Info   ] OPENXR Extension:   Name=XR_EXT_palm_pose SpecVersion=2
      [23:05:43.156][Info   ] OPENXR Extension:   Name=XR_EXT_active_action_set_priority SpecVersion=1
       
      While thankfully XR_FB_body_tracking, XR_FB_eye_tracking_social and XR_METAX1_simultaneous_hands_controllers_management are present on PC VR, 
      I'm still missing the crucial that offers (according go your own email you sent me a while ago on this topic) XR_EXT_eye_gaze_interaction which is by johnkearney 's own advice, more suitable for implementing dynamic foveated rendering, as it is an unfiltered / lower lag source of gaze vectors whereas XR_FB_eye_tracking_social is heavily smoothed / filtered for more stability and less jitteriness in order to be able to use it for social or gameplay purposes (ex menu item selection, or PC <-> PC or PC <-> NPC interactions)
       
      I really hope Meta will follow through on these requests for us poor PC VR devs who need a little help and support too.
       
      I've been a loyal Oculus user and dev since Day 1 in 2016 when Rift CV1 came out while I was at Felix and Paul, and have worked on a bunch of both PC VR and Quest games/apps since then, so I'd appreciate anything you can do to make all these extensions work on PC without developer mode and to add XR_EXT_eye_gaze_interaction which is still not even present with developer mode. I really need it to implement DFR in my engine effectively as other solutions that rely on XR_FB_eye_tracking_social are apparently not really worth enabling DFR at all on Quest Pro due to the latency, This is from multiple other project I've asked about, the performance gains are minimal whereas they would be much higher if the lag could be lowered, theoretically.
       
      BattleAxeVRAuthor
      Inspiring
      October 4, 2023

      https://twitter.com/BattleAxeVR/status/1709419768153878579

       

       

      I posted a build of hello_xr.exe for testing in the repo with full source code of the implementation in case if helps anyone out:

      https://github.com/BattleAxeVR/OpenXR-SDK-Source/tree/main

      It works on PC VR over Link too, great!

      BattleAxeVRAuthor
      Inspiring
      October 12, 2023

      It doesn't actually work correctly on Quest 3, which I just got yesterday, on standalone builds.

       

      It DOES work on Quest 3 and Pro over Link/AirLink, though. But on standalone mode, the extension is enumerated and returns that it's supported, as soon as I "resume" the feature, it bugs out the main controller poses which return invalid states and the buttons/joysticks don't work.

      BattleAxeVRAuthor
      Inspiring
      November 10, 2023

      By the way, simultaneous hands and controllers still doesn't work properly on v59 on standalone mode, only on Link/AirLink.

      I can't even get the v59 Android samples to build...we need some simple, step by step instructions that don't require a PHD or unlimited time to devote to it. I doubt anyone but me or maybe 1-2 other people out there have ever used the C++ code samples, given their utter lack of support here. Total waste of time...

      Terms & ConditionsAccessibility statement