VR Gloves (my project)

On the thought of magnets, it may be possible to use it for finger tracking, but using it to track arm/elbow movement probably wouldn't work. I have messed with hall effect sensors (sensors that detect the strength of any magnetic field) but I would guess that their range wouldn't be long enough to track full arm movement. Past that, they only track distance. This is done spherically as well, so you can tell distance, not position. I'm going to use an mpu (gyro and accelerometer in one) plus a potentiometer. Using inverse kinematics, I should be able to get pretty accurate position and orientation of the arm.

Towards size, I should be able to get it down to a size of around 1-2 inches square or so on custom made pcb. I've made my own arguing before in a 1in by 2in package with all pins (and extra pins for ground) so with only surface mount chips, I can get it pretty small. Prototypes are generally quite bulky and large. It's proof of concept. :D so it's not going to look pretty.

Now, towards priorities. I plan to go the order of elbow tilt/rotation/open, wrist tilt/rotation, fingers/thumb, getting supplies/funding for cheaper/better (better finger accuracy), build and test new prototype, make a dll to get information(driver).

I'm largely making this up as I go. I've never use i2c before, never made a driver, etc. Just try to look at it as logically as I can.

I typed this on my phone, so if there are wrong words or spelling, that's why.

I have an idea for virtual gloves to work along side with the glasses. Imagine finger tips inside of a glove containing not only sensors but vibration 6 points on each glove each finger tip and one center of palm. With an addition g sensor on back of hand. You could add one sensor to the top of the glasses to give you triangulation between your palm and both hands and your head. Knowing the porpotions of the human body you should be able to reach around yourself... ^^ but not so much touch your toes. The sensor point in your palm should give reference to the bending motion in fingers knowing you can't pull your finger tip in without bending the knuckle. Now the most difficult part really is being able to design the game around the simple concept of range of motion from the head sensor to the palm sensor in a 3d environment how far are you reaching. A simple sweat band with a sensor could also be used for an elbow but more wires would be involved and you would end up wearing sleeves. My idea the gloves would not give vibration to the front of your finger tip but the back giving your fingers still the ability to feel natural while holding a controller but being able to reach out when needed. Pair with blue tooth and some sort of battery pack on the back of your hand you got a sick glove.

Very cool idea. I actually was thinking the same over the last few days and am glad someone is working on it.

"Infamous" wrote:
I have an idea for virtual gloves to work along side with the glasses. Imagine finger tips inside of a glove containing not only sensors but vibration 6 points on each glove each finger tip and one center of palm. With an addition g sensor on back of hand. You could add one sensor to the top of the glasses to give you triangulation between your palm and both hands and your head. Knowing the porpotions of the human body you should be able to reach around yourself... ^^ but not so much touch your toes. The sensor point in your palm should give reference to the bending motion in fingers knowing you can't pull your finger tip in without bending the knuckle. Now the most difficult part really is being able to design the game around the simple concept of range of motion from the head sensor to the palm sensor in a 3d environment how far are you reaching. A simple sweat band with a sensor could also be used for an elbow but more wires would be involved and you would end up wearing sleeves. My idea the gloves would not give vibration to the front of your finger tip but the back giving your fingers still the ability to feel natural while holding a controller but being able to reach out when needed. Pair with blue tooth and some sort of battery pack on the back of your hand you got a sick glove.

This is pretty close to the idea I was having using magnets as sensors like what they use in a Wacom tablet pen or the sixense STEM. I really wonder why STEM doesn't have VR gloves as part of their system. Though, it's not like I have a good understanding of how STEM actually works.

Stem is just motion tracking. It tells angle (accelerometer) and applied force (gyro). I'm sure I'm over simplifying it, but the point is, stem doesn't have the extra bits, finger tracking being the main thing. I'm pretty sure they also don't include any form of gesture system, and it's mainly just data being pumped out. I can't say that for sure, as I don't have a stem system at all. Another reason not to use it, is price. On the kickstarter, 1 base with one stem pellet thing was $150, with no controller (ignoring early birds that is). That's just one point of data. I'd be hoping for around $150-200 (just a guess as final price) for the pair of gloves, and that'd be effectively 4 stem pellets (wrist and elbow for both arms), and that includes the extra stuff like finger tracking and buttons. Of course, this is all assuming the plans in my head for version 2 would work out. (potentiometers instead of flex sensors and a rigid glove instead of cloth like I currently have.

The problem with using magnets is mostly range and accuracy without interference. I haven't messed a lot with magnets and electronics, but magnets can only give you a distance, not really direction. If you were to use something like a compass, then it would be over powered or at the very least, distorted by earth's magnetic field. In my opinion (haven't not really messed with it), it's just not accurate enough.

O-o... I'm not trying to shut out any ideas people are posting (I know it may seem that way). Just giving out why I'm doing what I'm doing. :D Keep giving suggestions. Ways to improve it and make it better/cheaper are always better.



EDIT:
Just to go back to the stem system. Don't get me wrong, it's an amazing system, and you can do tricks like ik solvers to get close to accurate bends in the elbows and such. The problem I have is completely price. You can preorder the stem systems right now. To get the full package to use it to it's fullest, is $399.99. That's $100 more then the rift.

My goal is to be cheap. It's intended to be an accessory to the rift or any other vr headset. I don't think an accessory should be more expensive then the device it's is being an accessory to. I also know that the tech for stem is not nearly that expensive, especially when bought in bulk like they are definitely doing. You can get an accellerometer off of mouser.com for $0.56 if you buy 1000 or more, and I can promise you that sixense is not only buying more then that, but probably has insider deals to get it cheaper.

Sixense is not about coming out with a cheap product. They charge a lot. They are a good system, but they are just to expensive for my taste, knowing what is more then likely in them costs less then $5 when they are charging $50 per pellet. But hey, that's research and development for you. (though most of the R&D came from the razer hydra, so technically, there wasn't really a ton of cost with R&D specifically for the stem system.)

So, I decided to look up how wacom tablets work and I was quite surprised. http://www.tablet4u.co.uk/techinfo.html

I really don't know what makes the best tracker. Since you don't either the only thing I can suggest is to research it the best you can. I'm the hands on type, so I would probably buy one of every type of sensor I could afford and just play around with them until I found their limits.

One thing I do know is that you can use math to deduce a lot from just a few data points. It's fine to play around with different things in a prototype phase, I even encourage it greatly. However, at some point, I would try to figure out just how much tracking data is actually necessary and how much can be deduced with a few simple equations. That said, there will be a balance to be had between making the software (and any involved processors) do the heavy lifting versus having the glove do the work.

Either way, I'm sure I'm not the only one desperately waiting for a VR glove solution to appear to fill this gaping hole in my VR life.

I don't hate the idea of using something like the Hydra or STEM but I really do prefer the idea of using my actual hands as an interface since there are just so many more possibilities for interaction.

Edit: I just wanted to say I really agree with what you said in your edit. Cost, sadly, is a major concern. The more affordable it is the easier the adoption rate will be and the faster progress is made. I want to experience a virtual environment that impressed me as much as Crysis when it first came out, at least once in my life. It doesn't necessarily have to completely fool me but I would like to experience an environment real enough that I can "forget" that I'm in a virtual environment.

I also want to see AR take off. Something along the lines of wearing a smart phone and having an always present HUD as you experience your every day real life. This is an idea that's often toyed with in games and makes a great explanation for in-game elements like an in-game interface. I'd just like to see the idea make it to the real world. A good set of VR gloves could be just as useful in AR as it would be for VR. I know there's Google Glass but I get the feeling it'll have to go through a few iterations before it reaches the point that it surpasses tablets as a must have.

"CaliberMengsk" wrote:

The problem with using magnets is mostly range and accuracy without interference. I haven't messed a lot with magnets and electronics, but magnets can only give you a distance, not really direction. If you were to use something like a compass, then it would be over powered or at the very least, distorted by earth's magnetic field. In my opinion (haven't not really messed with it), it's just not accurate enough.

I'm a bit curious about that, would it be possible to triangulate their positions relative to each other or a static point(s)?

ok so I did some research on the whole idea here.
Basically the Oculus does head tracking in game.
A while back I heard about head tracking from the good old source of youtube and came across this.
http://johnnylee.net/projects/wii/

would it not be better to have physical head tracking applied at the same time as digital head tracking?
eg use wii sensor bar on the Oculus.
Led tracking for your fingers. (with wiimote stuff only 4 ''contollers'' at a time, but i''m sure that can be worked on)

Basically if you use the wii sensor and controllers in reverse you get physical head tracking. Applying it to Oculus would give you physical world sensing as well.

You could then apply things like the guns used on wii, bows, driving wheels etc. Opening up not just the glove accessory but also other physical tools.

Edit*
just thought of as well, because you can track your hands in game when they physically appear, you can get them rendered in game making it feel more real.

Yep, rendering the arm would be a final goal, but I doubt there will be a system that render's your own arm. I could see a system where you see yourself with a fully and complete virtual avatar. You could be shorter then you really are, or taller, and actually be placed visually like that. A pretty cool concept to step into someone else's shoes. You could be a dwarf and actually be at the height of one.

As toward the head tracking via wiimote, it's only really valid because of distance from the viewing plane. The wiimote system shown in the video is fine, and a great concept, but I'd wager a guess that in full practice, as you get closer to the camera, you start to reach the camera's max FOV, and the data points exceed the range. When it comes to head tracking, getting the points of reference out of the camera's range makes it impossible to tell what you are doing. As with anything light based, it must have line of sight to be able to work. Your hair, arms, etc. gets in the way and the whole system is thrown for a loop.

It may be a good idea to use something similar as a way to offset drift. Just have the IR led's on the glove, and the camera attached to the head. That said, it'd probably be hard to implement, and would add cost, and with an ideal motion tracking system, shouldn't be needed.

The rift has problems with drift because there is no point of reference for turning left and right as you can turn yourself 360 degrees, and gravity isn't applied sideways, so there is no way to get actual angle on the up and down axis. There's no force to say what angle you are at. So an infrared camera with infrared led's at specific points could not only handle positional tracking, but could also handle drift compensation when it's visible.

As stated though, that stuff adds to cost, and beyond that adds to complexity. People don't want complex systems they have to set up, they just want to use it and it to work.

Now, toward triangulation via magnets, yes, that can work just as well as triangulation in any other system. Though, you would have to use bare minimum of 4 points of reference, not 3. With just a triangle, you can only figure out position on 1 dimensional plane. So, for instance, you could get forward and backward, left and right from a triangulation system, but you can not get up and down at the same time. Because the 3 points of reference can only create 1 specific plane. Having a fourth point in the center, but above or below the plane of the 3 other points of reference and you can get altitude on that plane.

Magnets just seem like a tricky and over complicated and possibly expensive process as opposed to physical means like a volume knob type of thing. In the case of my glove, it's easy, bend the elbow based on the value output by a knob, rotated the arm based on the accelerometer, spin if needed by the gyro (in the case of the hand spining on it's y axis). In my head at least, it's a less complex process.

It could be easier, I don't know, but I do understand my system and why it should "work", so that's why I'm doing it that way. :P I just don't know enough about magnetic systems for this type of thing to say if it is or isn't possible, and since I have a way that is fairly sound in concept, why go for the extreme maybe one.


(I'm liking these little talks btw)

I'm really impressed with this project and i'll keep viewing this thread to see updates on this! I myself am more of a 3d/animation developer, so I get really amazed by the stuff you're doing, keep up the good work!