Forum Discussion

🚨 This forum is archived and read-only. To submit a forum post, please visit our new Developer Forum. 🚨
cloudcray's avatar
cloudcray
Honored Guest
12 years ago

Glove with buttons, joystick, EMG, and gesture recognition

Hey, guys - I've been working on an input device that's meant to be totally backwards-compatible with existing games, but still provide support for full avateering as new games and applications arise. It's got joysticks on both hands to control player movement, touch buttons for fast input (trigger, jumping, etc), and gesture recognition. It's also got an EMG sensor that can distinguish between "closing your hand" and "clenching a fist".

Here's a brief demo I put together with Skyrim. There's a little bit of lag, but the game is completely unmodded:
https://www.youtube.com/watch?v=MOYST-Uy8mo

Any suggestions? I know there are a few other hands-free folks out there, but none have really gotten the full effect I'm looking for in fantasy/adventure games. I'd like to offer this to a larger community, and I want to know if there are any must haves/nice to haves before I move on to the next prototype.

Any feedback would be appreciated :)

2 Replies

  • My 2¢ would be to ditch the thumb stick, buttons and basically everything except gesture recognition and focus on making the tracking as accurate as possible and the glove as affordable as possible.

    Pretty much anything you could accomplish with the buttons or thumb stick could be accomplished through good gesture recognition and maybe some props.

    Ultimately, I see some sort of pedal system being the ultimate way to walk/run while physically sitting. Though, having a set of gestures specifically for moving your character would be critical for anyone physically incapable of using pedals. In those cases, you could just see it as no different than an electric wheelchair; you're just going to have to accept that you're making a compromise on that.

    Between solid voice recognition and gesture recognition, you can do some rather amazing things controlling a character. Combining that with having your feet control movement, even when sitting down, can really hit the mark for complete immersion, outside of direct mind control.

    On a related note, I'm going to mention that the closer you can keep a pair of gloves under the $150 price tag, the more competitive it will be and more likely it will achieve a massive adoption rate. There's already at least a few groups working on a $300-350 model and at least one of those is using a very accurate but very expensive existing pair of gloves as the basis. That's why I mentioned aiming for a significantly lower price tag.

    If you're really determined to stick with having buttons and other such stuff, do it as an optional attachment. I really can't fathom the need for such attachments but I just might not be the intended audience. I want as much immersion as possible and stuff like buttons, joysticks and other props reduce that immersion compared to a fully functional and accurate VR glove with working gesture recognition.

    I'm at the point where I've been contemplating putting together a cheap prototype pair of VR gloves just to have something, anything to work with. The project I want to work on is really at a dead end without a pair of working VR gloves. I'd settle for mediocre, at best, gesture recognition, so long as it was workable. I seriously doubt I'm the only one in this particular boat. I feel like it would be very easy for gesture recognition to surpass voice recognition as an effective controller for both accuracy and ease of use. In fact, my project is wholly built on that premise.

    One last note, stuff like autohotkey have been doing the same type of thing for mapping keyboards and mice to controllers and vice versa. So, remapping gestures to other buttons and such is trivial compared to accurate gesture recognition. I'm not saying having the ability to remap stuff built-in wouldn't be cool but having poor gesture recognition just isn't going to cut it with what your competition is putting out.

    In any event, good luck with your device.
  • meyome's avatar
    meyome
    Honored Guest
    Wow, this is exactly the controller I wanted to see on the market

    The use of muscle recognition a first for me, i don't know any other controllers doing this, but it did look like you were putting a lot of effort into making it sense it. If it is too hard to make an action it would get tiring after only a short while so this needs to be, not trick-able, but not so hard that it's a nuance.

    Joysticks!!! someone finally understood. This for me is a major feature because having a joystick allows for longer gaming sessions that don't drain you physically. The open option to use heavy motion sensing while having the option of a less body intense play style would be great on days where you feel sick or tired.

    motion sensing is a major marking area for these VR headsets as well and it lead me to wonder, can it sense the muscle movements of each finger. if so, this would allow you to move small pieces on a chess board in a game or socializing environment so i would be very interested in how go the muscle detection can get.

    Love the product and look forward to were it goes :)