cancel
Showing results for 
Search instead for 
Did you mean: 

Cardboard+ARToolkit Unity plugin - optical hand-tracking

konstantin_loze
Explorer
Hi to everyone. I have already promised a few times in the Cardboard Reddit channel to upload a youtube video of my experiments with QR code markers in combination with Google Cardboard. Well, I did not really have time to make proper ones yet, but I decided to just take two 3-4 minute clips with an old smartphone of mine, holding Samsung Galaxy S4 in my left hand and the QR tracked object in the right hand. As you can see, the lighting conditions were very, very poor for this type of tracking and especially on the second video you can barely see anything, also there is a buzzing sound in the background from the neon lights. Sorry about all that. It all looks very much like the recent hit on the Google Playstore called "Germbuster". The difference is this is an entirely open unity project that you can build on and improve (e.g. make a bigger marker, etc.). For example, in the Gravity Gun video, I used one of those 5 EUR bluetooth keychain gamepads to have the basic functionality of mouse left and right button. There is a lot, lot more to do, but I decided it is best to share with the community already at that early stage. Links to the APKs and to the Unity Project will follow. Now, here are the links: https://www.youtube.com/watch?v=5D6wXBYsaio and https://www.youtube.com/watch?v=04GLwA4GnB8. Enjoy and ask questions!
2 REPLIES 2

MikeF
Trustee
Very cool!
how well does the tracking hold up when head movement comes in to play?

I'd be very interested in checking out the unity project

konstantin_loze
Explorer
"MikeF" wrote:
Very cool!
how well does the tracking hold up when head movement comes in to play?

I'd be very interested in checking out the unity project

Well, it is the same as if the gun moves - if you move your gun vs head position faster - tracking gets lost at a certain point since the image is smeared due to low frequency of capturing. If you move your head and hand in parallel, like in aiming down the sights, the results are actually not bad at all. It depends on the size of the marker, lighting conditions and distance from the camera (the shorter your hand - the better since then the marker travels shorter distance per angular displacement). The 4x6 multimarker setup is the best so far in my experiments because the individual markers are relatively large, you need only one visible marker to hold on to tracking, so occlusion issues are alleviated and if more individual markers are visible, the tracking is much more robust, even if you move faster.
Now, on the Cardboard reddit a member suggested using FishEye lens, but I am not sure how to calibrate the Unity Plugin for the huge distortion introduced by those lenses. I will buy some of those cheap clip-on sets and experiment, but maybe a safer bet would be 0.4 wide-view lens, which are often included in the clip-on sets anyway. Increasing the trackable FOV can result in either alleviating occlusion issues OR more robust tracking when moving faster by allowing larger markers.
Will post soon the project so that anyone can experiment themselves.