Skip to main content
Up and Coming
June 15, 2013

Unified Head and Hand Oculus Positional Tracking

First off, let me say I have no idea HOW this could. I just have an idea that someone should figure it out!

Oculus is working hard on positional tracking for the head, but should they be focusing on hands at the same time? Right now there are all of these different ideas for controllers/gloves/motion tracking. What if you could use any controller or device you want and STILL have positional tracking for your hands that are ALSO relative to your head.

I got to try the Rift with the Hydra, and one frustrating part of that experience was that my virtual hands in the Tuscany demo were not exactly where my real life hands were. I'm sure there are several reasons why this is the case, but the reason I can think of is that the Rift wasn't be tracked, so where my head is located relative to where my hands are located isn't matching up. If the same positional tracking was being used for the Rift and my hands, however, I imagine it would be very easy to match up the hands being exactly where they are supposed to be in relation to my head. But how do you do this and still allow the use of any controller? My idea is positional wristbands.

You could put a wristband on each wrist, making sure that the logo lines up with the center of your wrist. These could be wireless or there could be long wires that plug in directly to the Rift. Assuming that whatever positional tracking solution for the Rift could also work on these bands, you could track the head and both hands in virtual space and in relation to each other. The bands could even track rotation and other movements of the hand. The best part is that this still leaves the hands free to use the input device of you/the games' choice.

Using this method, you could aim with an Xbox controller. You could have a game that could combine the Leap Motion and Oculus' positional tracking. You could still use the Hydra. You could use a glove. Or you could just use the bands (if it were, say, a boxing game). It's about creating an Oculus standard - all Rift compatible games can take advantage of positional tracking for head and hands, BUT it still allows the flexibility for a game to choose different input devices (you could even play mouse and keyboard and just fling your hand around for specific commands or to point at menu options).

Perhaps this idea has already been floating around. Or perhaps the next post will tell me why it's not possible. It's just an idea that, in as much daydreaming about the Rift as I do, I hadn't thought of before, so I thought I would share. Thoughts?

    59 replies

    Participant
    June 18, 2013
    "Tbone" wrote:
    One issue is that Hydra's current solution has way too much latency, not only for the Rift, but for the controllers as well. I don't know why that's the case. Can the magnetic sensors be tweaked? Or is that system just flawed by design?


    Hmm... I never noticed much latency. I thought that was the point of being wired. Most of the complaints I've heard are about accuracy. It could just be me, though.

    Having tried out the Rift a few times now, I can tell you that one of my pet peeves is still having to use the right analog stick to rotate left and right. It's a major nausea-inducer, and I'm not sure that any amount of resolution and positional tracking will fix that. The main solution is to make sure the Rift is 360 degrees compatible, so that all turning can be done naturally. For this reason, I can't see optical tracking being a valid solution - it just creates more issues than it eliminates. Get a chair that spins now!


    I totally agree that turning with the analog stick is a problem, and the long term solution is being able to turn yourself 360 degrees. That just seems far off to me, I guess. The Rift would need to be wireless, people would have to change their behavior, etc. This is why in my "Position Crisis" demo, I picked gameplay that doesn't require any turning at all! Obviously that's not a real solution either. I just don't see a great way to get there, without a lot changing.
    Up and Coming
    June 18, 2013
    A couple of thoughts:

    1) Does the Hydra really have such high latency or is there a lot of filtering going on in the SDK in order to reduce noise in motion? Such filtering would feel like latency but is actually smoothing.

    2) Does latency in hand motions matter as much as it does for head motion? I was a bit annoyed by the lag in my hand motions when messing with the Tuscany/Hydra demo, but I was also annoyed by a lot of other things, so I cannot draw a conclusion.
    (╯°□°)╯︵┻━┻
    TboneAuthor
    Up and Coming
    June 18, 2013
    It depends on what you want to do. A good example is throwing a ball. As I tried throwing a ball in Tuscany, it proved difficult. What we discovered is that the key was to release the trigger AFTER you think you should, due to the latency. So whenever there is something involving physics, the latency becomes more noticeable. It's been a while since I've demoed the Hydra, so I can't remember how bad the latency really was. I do know that if you watch a video with someone using the Hydra, you can see the latency frame by frame. Just look at the video of the guy who was using the Hydra for head tracking as well as the firearm and you can see it lags behind just a bit.

    I will say the bigger issue is that your hand isn't where you think it should be. Getting unified positional tracking for your head and your hand should correct that, though (I think), but to do that, the positional tracking needs to have as low latency as possible.
    The Furious Angels - an Oculus Rift Online Gaming Community!
    Up and Coming
    June 18, 2013
    "Tbone" wrote:
    ... I will say the bigger issue is that your hand isn't where you think it should be. ...


    This is something I've had experience with before in making interfaces that are controlled with the mouse. I think in terms of VR that it is more important to reduce latency/smoothing than to make sure that your hand is exactly one-to-one where it should be. You don't need to know exactly where your real hand is, and your virtual hand doesn't need to be positioned just like your real hand, but your virtual hand needs to respond quickly.

    I'm also not convinced that the Hydra's problem is latency because it may be due to filtering and smoothing. There is a subtle but important difference. It just feels more like the Hydra is over-filtering the input than that there is lag.
    (╯°□°)╯︵┻━┻
    Participant
    June 18, 2013
    "Tbone" wrote:
    Just look at the video of the guy who was using the Hydra for head tracking as well as the firearm and you can see it lags behind just a bit.


    Which, mine? My video capture wasn't great, I think it may have made it look worse than it is.

    I will say the bigger issue is that your hand isn't where you think it should be. Getting unified positional tracking for your head and your hand should correct that, though (I think), but to do that, the positional tracking needs to have as low latency as possible.


    I haven't tuned it perfectly, but, for example, I can pick up the Hydra wand off my desk by reaching for where the gun is on screen.

    You guys should just try my demo; the situation may be better than you think. ;)
    Up and Coming
    June 18, 2013
    "eshan" wrote:
    You guys should just try my demo; the situation may be better than you think. ;)


    I'll do that as soon as I can. I'm not pessimistic about it. I was even going to say that unless a person made sure to edit and sync video properly then it is not really not possible to tell from a video how much lag there is. I've even seen people watch the same video and come to opposite conclusions about this kind of thing!
    (╯°□°)╯︵┻━┻
    Participant
    June 18, 2013
    "jwilkins" wrote:
    This would be interesting because the Kinect uses structured light, but I've never got confirmation on what the LEAP Motion uses. Two structured light patterns overlapping would probably not work well, but neither could it be good for the the 6 IR LEDs in the LEAP Motion to be shining into the Kinect.

    So, I'm really curious if these two devices can get along.


    Excellent question. I'll try to do some tests to check this.
    Inspiring
    June 18, 2013
    Btw if you have latency issues with the Hydra, have you tried using the Sixense drivers instead of the stock drivers?
    Up and Coming
    June 18, 2013
    "sven" wrote:
    Btw if you have latency issues with the Hydra, have you tried using the Sixense drivers instead of the stock drivers?


    Thanks for making me aware there is an alternative.

    I'm also wondering if there would be a way, besides backwards engineering the raw HID input, to get unfiltered data from the Hydra. (and even then the "raw" hid data might be filtered...).

    Are any of these drivers open source?
    (╯°□°)╯︵┻━┻
    TboneAuthor
    Up and Coming
    June 19, 2013
    Next time I do a Rift demo, I'll try to do something with the Hydra as well and see how bad or not bad the latency is. Is that sort of tracking good enough for head tracking, though?
    The Furious Angels - an Oculus Rift Online Gaming Community!