MS Kinect as Input Device

Have you tested this:

OpenNI (standard framework for 3D sensing) API and 3D hand tracking library and framework uses OpenCL or CUDA:
http://www.openni.org/files/3d-hand-tracking-library
http://cvrlcode.ics.forth.gr/handtracking

zigfu development kit for Unity 3D works with OpenNI:
http://www.openni.org/files/zdk-for-unity3d

http://www.youtube.com/watch?v=N3ffgj1bBGw

"LaurentMT" wrote:
Hi,

I'm also interested in investigating this kind of usage (Rift + Kinect).
I know Kinect is not very popular in the VR community because of the latency but I still want to test it farther.

I already started a project on the subject during my spare time. First results with Kinect alone are quite positive to my own opinion. Now, I have to wait for the arrival of my dev kit (surely not before may or june, since I'm a preorder) but I try to anticipate the eventual problems to do fusion of data coming from Kinect and Rift.

If I can get some good results, I'll share work and sources with everybody.

I'll keep you in touch.

Laurent

I haven't tested these solutions yet but some could be interesting (the hand tracking is quite impressive).

I had to pause this work for a while (unexpected long stay at hospital) but I will continue as soon as possible. At the moment, my priority is the lean based navigation system. The navigation works fine but I noticed a delay induced by skeleton detection : It takes around 20-30ms to have a skeleton returned by the ms kinect sdk.
The library that I use calls the sdk into the game loop (in Update method). So, using this method FPS is around 30fps at its best which is far from the target 60fps required for VR.
According to me, computation of skeletons should be parallelized with game loop in order to reduce this latency.
I have started to rewrite the library to bypass this limit. Still a work in progress but I've done some progress.

I finally received notice that my Rift is being processed for shipping. So, hopefully I will be able to do some actual development with the Rift and my Kinects soon. I had to take a break from dreaming as I was getting too frustrated that I did not have a Rift in hand.

I bought this for my personal use, but everyone in my scientific visualization research lab is anxiously awaiting me bringing my Rift to the lab.

I just got my Kinect in last night. Using zigfu's Kinect Unity3D package, it only took 15-minutes to get a demo up-and-running. I wanted to add the Razer Hydra; but for some reason the sixense drivers aren't playing well. (I think I need Unity Pro).

Looks awesome! You were able to lean back pretty good without losing balance/orientation, so I guess it felt quite real :lol:. How was the latency, did you notice it?

I received my Oculus Rift DK a couple of weeks ago. I then bought Unity Pro and have been having a blast doing C# scripted procedural geometries and outer space simulation scenes. I need to put the Kinect on my list of things to play with! I like what I am seeing here.

Hi everybody,

Since my last post, I've made some good progresses.
I've adapted the kinect library in order to keep a good framerate and to add a few things. For now, I've got : a lean based input controller, avatareering and a kind of "neck tracking" (in order to synchronize the position of the OvrCameras with the avatar). All this stuff is still very rough and need more work but it's already fun and provides quite good results.

I received my Rift on monday. In order to test the integration of the Rift with Kinect, I thought it would be fun to get a kind of hoverboard controller (yes, like Marty Mc Fly :roll: ).

Here's the result of my first tests.




To be honest, the first sessions were like learning again to ride a bicycle or do skiing. But the very positive result is that even if the controller needs to be really improved, I feel less motion sickness during these tests than when I try some demos while seated.

Second test of the hoverboard controller in the wild.
The environment is 25 square kilometers of moutains, rifts, cliffs, dunes, ...
Perfect environment to cure your vertigo !
It's also my first try to build a scene with Unity. I'm a real noob, so it's low poly and textures are :shock:

Cool. Looking better.

I'm developing an application which uses Unity3D + Oculus + Kinect (the Holy Trinity, FTW!), but when I use the Kinect the framerate gets limited to 30, which makes the Oculus experience subpar. It's not a performance issue. So my question is:

How did you adapt the Kinect library in order to achieve a better framerate?

Any help will be greatly appreciated!