Skip to main content
BattleAxeVR
Inspiring
August 1, 2024

"Simultaneous hands and controllers cannot be enabled while Body API is active"

  • August 1, 2024
  • 6 replies
  • 3175 views

Hello Meta, I would like to report an issue with your OpenXR runtime and Quest functionality.

In my game, I rely on Body Tracking as a standard feature that's supported on all Quest headsets, but would also like to use multi-modal mode with controllers when available, in order to use Controllers (or Logitech MX Ink) at the same time as not only hand tracking, but body tracking as well. Seeing your fingers wiggle around accurately is really nice, for gestures and realism.

 

Since hands are a part of the body, and finger joints are tracked just fine when I put the controllers down and it switches modes, I don't see why, technically, multi-modal would only work for hand tracking EXT (without body tracking) and not the full body API which includes hand tracking.

I reported this issue before but was told it wasn't a good place to do it:

https://github.com/facebook/igl/issues/117

Is there any way we can get body tracking and controllers to work at the same time?

 

At least on Quest 3 and possibly Quest Pro (or even Quest 2 when using Touch Pro controllers)?

      6 replies

      BattleAxeVRAuthor
      Inspiring
      August 1, 2024
      Aside: I'm sure I'm not the only one who would like to use body tracking and controllers at the same time, and for example, I just received the Logitech MX Ink which is technically a controller, and I'm sure many devs and studios making social VR painting apps would appreciate having their bodies represented faithfully while also being able to use the stylus device. 
       
      So fixing this issue here would fix it for Logitech's new pen as well, which works fantastic btw. 
        BattleAxeVRAuthor
        Inspiring
        August 7, 2024

        johnkearney Hi John, hope you're doing well. I see you're still around these parts. 

         

        Can you take a look at this issue? I have several apps that are in flight now for both myself and my clients that require body tracking, but would benefit from multimodal support (1 controller + 1 free hand).

         

        This includes the new Logitech MX Ink which I have, but sadly it doesn't allow one Touch controller and the stylus to be used at the same time, which is disappointing. Anyway, I really need body tracking and being able to see my hands / forearms at the same time as use this new Pen for a new app, can you guys / gals take a look at fixing this issue with mutual exclusivity of multi-modal vs FBT? It's really limiting me.

         

        I'd also love to be able to pair two MX Inks at the same time and do various fun things with a second player on a 2D flatscreen, for example (collaborating. not everyone wants to be in VR). There are also plenty of fun gameplay possibilities aside from art apps, like assymmetric gameplay (remember Wii U?). An extra player can tap on the screen to drop new platforms the other players can jump onto.

          BattleAxeVRAuthor
          Inspiring
          September 26, 2024

          Hi neurodivrgent , can anyone at Meta confirm whether this but will be fixed? 

          I just tested my Logitech MX Ink stylus in my app but I really need FBT and finger tracking at the same time for immersion. It's just awful looking that the fingers aren't moving when you're painting stuff. 

            Mikyjax
            Inspiring
            November 13, 2024

            BattleAxeVR   Hello Sir!

            I just bought the mx ink and realized that you cannot have your left hand and the stylus (in the right hand) at the same time. This is in its state useless for my app as most of the action are executed by the left hand. I just needed the stylus for precise control in right hand. Did you find a work around or had any answer from meta?

            Thanks!

            BattleAxeVRAuthor
            Inspiring
            November 14, 2024

            They did add in v69 some improvements for the MX Ink, basically at this point you can pair it instead of either the left or right controller and still use the original other one, as is, without any code changes (unless you want or need to reassign). I haven't tried the simultaneous hands and controllers multi-modal yet, but I assume it works  too (but only on Quest 3). 

            My issue here is the Body Tracking with Hand Tracking both cannot be active at the same time, which means the controllers can't be used at the same time as body tracking, even though body tracking includes hand tracking. None of it makes any obvious sense to me, it seems like a needless restriction that they'll probably fix one day, when one of their own 1st party apps tries to do something with 1 controller (or stylus) and body tracking for greater immersion. You really want to see those fingers move properly when holding the MX Ink. I honestly do not understand why any of these restrictions are acceptable or even allowed at Meta. They're harming their own products and developers having such a "Bingo" chart of what accessories and features will work with one another.

            CarlosEngler
            New Participant
            April 4, 2025

            I'm currently working with a MX Ink, while with an sdk > 70 it is now possible to use controllers and hand tracking simultaneously, when touching the pen it instantly shuts off. I would like to know if any of you guys have succeeded in using both the pen and body tracking at the same time.

            Also, i would like to know if any of you were able to pair 2 pens at once? it looks like there is a lock when trying to connect more than one

            Terms & ConditionsAccessibility statement