Forum Discussion
Hybrid hydra + Wiimote
In reading comments about the accuracy of the hydra, and inferring a few points from it's SDK, I came to an interesting thought.
Obviously, long-term it's impractical, but technically it could be built as a hybrid device...
But here's the basic thing; The hydra appears to use some kind of magnetic tracking. While precise, it can apparently do some strange things, like being (in the words of John Carmack) as much of 15 degrees out of alignment after certain kinds of motions. - it does not appear to have any tracking other than it's magnetic system.
The Wii Remote plus on the other hand (that's the upgraded version with a gyroscope built in) has a reasonably precise gyroscope, and an accelerometer, as well as an IR camera.
The gyroscope can track rotation pretty well, and the accelerometer, when used together with the gyroscope, can give a very rough estimate of which direction the controller is moving.
The accelerometer also does a reasonable job of calculating 2 axes of rotation relative to gravity.
However, the positional tracking of a wiimote is absolutely terrible, while the hydra's is better, but by all accounts seems to have unreliable rotational tracking. (I think you can see where I'm headed with this...)
A final note about the wii remote is that using a modified sensor bar design with 4 led clusters instead of Nintendo's standard 2, it's been demonstrated in the past that you can track the remote's position relatively well.
However, since the IR camera is on the front of the remote, and has a field of view of all of about 30 degrees, in most situations it's difficult to keep it pointed at the sensor bar. (Using multiple sensor bars is theoretically possible, but you'd have no way to tell them apart.)
So... Here's what I was thinking; Create a rig that would allow a wii remote to be strapped to a hydra controller.
The Wii remote would be able to calibrate the hydra simply by being able to tell the approximate orientation, and comparing this to that of the hydra to see which of the two reported positions is correct.
The gyroscope would in general produce more accurate rotation data than the hydra, but you can obviously fuse the results. The hydra data can then in turn help to keep the gyroscope calibrated.
On top of this, you can set up the IR camera to track an external IR marker, which gives additional position and orientation data should the remote be pointing in the right direction.
If it is, you can fuse it with the hydra's position as well as recalibrating the gyroscope.
If not, you rely on the hydra's position data by itself. (though obviously, you can use the accelerometer and gyroscope readings to sanity check the motions the hydra is reporting.)
Maybe this won't improve things much, but given what I've heard the hydra's issues are, it seems like it would actually help quite a bit.
(Before you ask though; if you're using the hydra to track head movements for some reason, don't bother adding a wii remote to it as well; the headset itself can already do most of the things a wii remote can, and it appears to be better at it for good measure.)
As a hack, it could produce hand-held controllers that are slightly unwieldy.
As a commercial product however, (assuming it works), it would be as simple as integrating a gyroscope and accelerometer into the existing hydra designs. (and, potentially, depending on how much it helps, an IR tracking camera too.)
If it does anything to improve accuracy then, implementing a more practical equivalent to it should be fairly trivial by manufacturing standards. (That's not to say anyone would actually make such a thing though...)
Obviously, long-term it's impractical, but technically it could be built as a hybrid device...
But here's the basic thing; The hydra appears to use some kind of magnetic tracking. While precise, it can apparently do some strange things, like being (in the words of John Carmack) as much of 15 degrees out of alignment after certain kinds of motions. - it does not appear to have any tracking other than it's magnetic system.
The Wii Remote plus on the other hand (that's the upgraded version with a gyroscope built in) has a reasonably precise gyroscope, and an accelerometer, as well as an IR camera.
The gyroscope can track rotation pretty well, and the accelerometer, when used together with the gyroscope, can give a very rough estimate of which direction the controller is moving.
The accelerometer also does a reasonable job of calculating 2 axes of rotation relative to gravity.
However, the positional tracking of a wiimote is absolutely terrible, while the hydra's is better, but by all accounts seems to have unreliable rotational tracking. (I think you can see where I'm headed with this...)
A final note about the wii remote is that using a modified sensor bar design with 4 led clusters instead of Nintendo's standard 2, it's been demonstrated in the past that you can track the remote's position relatively well.
However, since the IR camera is on the front of the remote, and has a field of view of all of about 30 degrees, in most situations it's difficult to keep it pointed at the sensor bar. (Using multiple sensor bars is theoretically possible, but you'd have no way to tell them apart.)
So... Here's what I was thinking; Create a rig that would allow a wii remote to be strapped to a hydra controller.
The Wii remote would be able to calibrate the hydra simply by being able to tell the approximate orientation, and comparing this to that of the hydra to see which of the two reported positions is correct.
The gyroscope would in general produce more accurate rotation data than the hydra, but you can obviously fuse the results. The hydra data can then in turn help to keep the gyroscope calibrated.
On top of this, you can set up the IR camera to track an external IR marker, which gives additional position and orientation data should the remote be pointing in the right direction.
If it is, you can fuse it with the hydra's position as well as recalibrating the gyroscope.
If not, you rely on the hydra's position data by itself. (though obviously, you can use the accelerometer and gyroscope readings to sanity check the motions the hydra is reporting.)
Maybe this won't improve things much, but given what I've heard the hydra's issues are, it seems like it would actually help quite a bit.
(Before you ask though; if you're using the hydra to track head movements for some reason, don't bother adding a wii remote to it as well; the headset itself can already do most of the things a wii remote can, and it appears to be better at it for good measure.)
As a hack, it could produce hand-held controllers that are slightly unwieldy.
As a commercial product however, (assuming it works), it would be as simple as integrating a gyroscope and accelerometer into the existing hydra designs. (and, potentially, depending on how much it helps, an IR tracking camera too.)
If it does anything to improve accuracy then, implementing a more practical equivalent to it should be fairly trivial by manufacturing standards. (That's not to say anyone would actually make such a thing though...)
4 Replies
- That is an interesting idea.
My Hydra is on back order, so it will be a while before I get to see for myself what it can/can't do.
However, I would like to use the wiimote for positional tracking of the head. I have dabbled with using the wiimote on PC in the past, but so far I have not integrated it into a game. I would be interested if any of the Game SDK's which work nicely with it. - I also have a Razer hydra on back order and can't do any testing yet. The idea in the first post is valid I think. I'm guessing that the next generation of motion controlers will be a hybrid of 2 systems. The first one is a low latency tracking system that has issues with drifting, but works independently and doesn't have to be aligined with a camera or other system like that. The second system would have 100% correct tracking, but might have issues with latency and alignment. The purpose of the second system would be to just correct the first one from time to time.
- My main problem with the hydra is the horrible jitter that the controllers got when more than two feet from the base station (becoming completely unusable by 3 feet). Using the data from a wiimote you could use a smoothed out hydra absolute position combined with wiimote accelerometer data for relative position changes. That could possibly work well.
However, I would like to use the wiimote for positional tracking of the head. I have dabbled with using the wiimote on PC in the past, but so far I have not integrated it into a game.
Some of my students made a game a few years ago that used a wiimote for head tracking. They wore a pair of safety glasses with IR lights mounted on either side (the glasses came with the lights built in, we replaced the visible leds with IR). The wiimote tracked their head position (in 2d) and used it to control the view in a time crisis style rail shooter. A second wiimote was used as a gun. Worked great on a projector. :) - Mmm. 2d positional tracking with a Wii remote is a well established idea, (it's how the Wii's onscreen cursor works after all), but I've seen at least one demonstration of 3d tracking with it too.
Now, the standard Wii sensor bar has 2 LED clusters (10 leds technically, but to the system it mostly looks like 2 points).
This gives you enough information to track movement in a 2d plane, and rotation around the axis (the Wii tracks controller movement, but you can of course reverse the setup and track a moving marker instead.)
It can also sort of track depth based on several features. The distance between the two led clusters can give you depth information, but unfortunately, it's ambiguous, because the angle along the plane of the LED's also causes the distance to go down, so what you actually have is a 'position' somewhere along an elipse relating the angle and distance between the remote and marker.
There are two ways of solving this; One I've seen demonstrated ages ago, is to use a marker with 4 LED clusters instead of 2, which, if arranged correctly, gives enough information to correctly deduce both angle and distance.
The other, is possible in theory, but I've never seen it implemented.
The Wii's IR sensor has 3 data modes. In the simplest, you get x and y coordinates for each point being tracked. The second mode gives you intensity values for the LED's as well (but only 2 bits worth - 16 values). This can be used because if the controller is at an angle to the LED's the relative intensity will be different, but if it's head-on, both will be at the same intensity.
Finally, there is a 3rd mode which gives an 8 bit intensity value, as well as a bounding box for the LED clusters. Both of which can provide extra information useful for separating depth from angle.
But... No existing Wii driver on PC that I know of currently lets you use this third mode.
And tracking 4 LED's is probably a simpler and more reliable solution overall.
Anyway, moving on to the hydra, now that I have one, I've got a better sense of how it behaves. Mine behaves pretty well all things considered, but I can see it do things which aren't quite right. Sometimes the position is off, but the most obvious thing that happens is huge errors in angular measurements.
This is especially obvious if I move the controllers near my monitors. (they're old CRT monitors, so they use magnetic deflection to control the electron beam).
Another thing I've noticed, which is more to do with having read about how the hydra works, is that although it's configured in the drivers to give you a position as a set of cartesian coordinates, along with an orientation vector...
The way it works is quite clearly based on a mechanism that would more naturally be represented as a set of polar coordinates.
Given the physics involved, that also means that barring any other issues, the device will be progressively less accurate as the distance from the base station increases.
(the ability to detect the angle to the base station is limited by the electronics, and since the position is actually derived from the angle, the further away the controller is, the larger the jump in position for a given change in angle.)
... I wonder if the hydra SDK can output polar coordinates at all? It would make the accuracy limitations somewhat more obvious...
