Should Oculus bundle motion controllers with every Rift?

Anything that does not involve putting a controller or glove in your hand whether it works by sonar or lidar is likely to have the same latency problems that the the Kinect has due to the difficulty of object recognition, human features are not as well defined as QR codes. Glove with good positional tracking could be interesting but they would have to come in different sizes, and all attempts at tactile feedback on gloves start looking bulky and expensive. There are also allot of games where an empty handed gesture is a poor substitute to pressing a button or pulling a trigger. So I expect the standard VR interaction will be Hydra style pistol grips for a while to come.

The magnetic tracking used by Hydra/Stem is less affected by occlusion but has limited accuracy. Prio VR and Control VR strap lots of gyros to you and use inverse skeletal animation to figure out position, not affected by occlusion at all but even less accurate with no absolute positioning also involves lots of strapping up and are expensive. The DK2 camera is much more accurate than the PS Move and its low bit monochrome IR camera could be made both quicker and higher res in future versions for larger more accurate tracking volumes. Extra camera could reduce occlusion and make larger more accurate tracking volumes through stereo parallax. A couple of pistol grips with LED patterns on the knuckle guards would make a good compliment to the DK2s tracking system, would be both cheaper and more accurate than the competitors, cant see why it would cost much more to manufacture than a cheap game pad.

Pistol grip controllers could be made a bit more versatile with a palm strap and a trigger for each finger with haptic and force feedback which could simulate picking up more general objects. Could also have haptic buttons that raise out of the sides and back of the grip when programmed.

Another interesting technology that seems under researched is radio frequency position tracking which could be as accurate as optical tracking but less affected by occlusion.

A balance board could be a good input for movement and rotation even when sitting.

If Oculus dont include a controller than I expect 90% of Rift users will just be using mouse keyboard and a few game pads. Things like Stem will likely remain a niche market especially if the price stays so high.

Personally I'd prefer they not include anything, mostly because whatever they'd include would

* Make it (the Rift) cost more
* Probably not be robust enough for anything other than cockpit/racing games, due to the nature of Oculus' focus on a seated experience
* Since they'd cost more, it'd discourage interest in better controller solutions that aren't limited by Oculus' liability concerns for the Rift
* They'd probably make it an optical system, which, even if they are more accurate than magnetic ones, suck because of the occlusion problem, which is particularly bad for something which you hold in your hand. It also introduces even more software overhead to an already hefty head tracking overhead.

I'd rather develop for robust 3rd party devices than be restricted to a barebones one because Oculus bundled it with the Rift and sucked up peoples' peripheral budgets with it.

tl;dr: You need to remove the friction between your feet and the ground if you are to make a foot-controlled locomotion controller. You need a foot-controller locomotion controller if you want to have the hands free for natural input. What do Oculus? What do?

I'm going to be pretty disappointed if Oculus come out with optical-based controllers. It's just not robust enough to have universal applicability for 3D input.
I think the single biggest issue when designing an ideal VR control scheme is the problem of simultaneously having free hands for input, and the capacity for locomotion. Because if you're not using your hands to move a thumbstick, you have to be using your legs to walk around in VR. But for this to feel natural (and by this I mean, to make it so you're not just walking on the spot feeling like an idiot), you have to be making real footstep motions. But the friction between your feet and the ground will make you go forwards, so there are two options:
1. Remove the friction force (i.e. reduce the normal component of friction to zero by suspending the player in the air, ala Omni). Problem with this is any system that physically lifts you off the surface you are 'running' on is going to be cumbersome and big
2. Move the actual ground underneath the player (i.e. a 'traditional' treadmill that moves in a direction opposite to the one the player is trying to walk in). This makes the most intuitive sense, but obviously poses safety risks, would be expensive, and would be similarly cumbersome

Both options kind of suck, but there's no other way around it. And if we want to have the ability for our hands to be free ever in VR, then this is a problem that's going to need to be solved. If Oculus wants to make an ideal 3D input system to be used with CV1, then it's a problem that's going to need to be solved soon.

edit: A thought just occurred to me - Oculus wants to make a great input device, but they're also sticking to their guns about the whole 'seated experience' thing. Since you can't walk convincingly while sitting down, I think that I'm going to end up disappointed when they announce their controller setup. :(

"PatimPatam" wrote:
After some of the recent developments i think it's getting more and more likely that we will learn about OculusVR controllers fairly soon..

Well, after the dust has settled down from last week i have to admit that i was wrong.. However i did get the impression at Connect that Oculus would have liked to show something, but they were probably not satisfied with their current prototype and didn't feel it was ready for show time. I did try the Crescent Bay HMD, and as impressed as i was with the visuals one thing that i noticed is that being able to see your own hands and interacting with the environment is more important now than ever before.

I totally understand Oculus not wanting to show or talk about something that they are not 100% sure is going to work, since that would only create more confusion and speculation. Having said that, personally i would love to know the things that they have tried and what are the exact issues that they are facing..

VR is all about making compromises, there is no perfect solution to each and every problem (especially regarding input), but for me a good compromise is better than no compromise at all.


From a functional point of view i still believe that having a 6DOF tracked hand controller with an analog trigger for each finger would strike a good balance between the convenience, physicality and instant feedback of a traditional gamepad, and the realism and richness of interaction provided by gloves or something like the LEAP motion. A simple strap going around the back of your hand would suffice to keep it in place even when not pressing any "trigger". I think these triggers wouldn't need to "push back" but ideally they should be able to stop movement or "lock" at any position. Maybe they could even be a bit concave, wrapping around your fingers and detect a bit of lateral movement as well.

With something like this you could closely simulate the feeling of holding a sword, a gun, or a tennis racket (with variable thickness depending on where the triggers lock), and get that physical feeling not possible with a glove or pure optical tracking.

At the same time you could simulate moving your hands, grabbing things, opening doors, or even "touching" objects (for instance if you had your palm open against a virtual wall, the device could prevent you from closing your fingers; if you had your palm open to hold a virtual glass the device could lock the buttons matching the shape of the glass).

Yes probably it wouldn't be good enough to trick your brain into thinking they are really your own hands, and it wouldn't be good enough to perfectly simulate complex interactions like playing a guitar or operating on a patient, but even in those extreme cases i believe you could still simulate some pretty decent approximations.


From a technical point of view the new Crescent Bay prototype shows that we now have a big enough tracking volume for hands in a seated or standing still experience. Latency should be the same as the latency for the HMD (sans the prediction), which i think should be more than good enough as well.

If occlusion is too troublesome to sort out with only the existing front camera, a 2nd inside-out camera with a wide FOV could be used on the headset to help track the controllers. If the HMD is blocked by the controllers, the relative position between the external camera and the HMD could be calculated integrating the relative position between camera and controllers and the relative position between controllers and HMD.

Maybe this inside-out camera could be used at the same time for pass-through functionality similar to Gear VR. Obviously this would add both weight and cost to the HMD, but i don't see it as a show-stopper.


Anyway I'm still hopeful that we will see something by next CES or GDC!

Rant over!

What I have been trying is this: Keeping the Leap attached to the Oculus, and in V2 this seems to work much better for tracking that being on a table, but it does loose tracking, but that is mostly with arm extended, and there are posture of the hands and arms that the Leap tracking still doesnt like much. So I have been trying to integrate openCV with it. One thing I noticed is that OpenCV instantly recognized the Leap Camera. I have not yet been able to get an Image trough openCV, but I think I am close. I am hoping to leverage openCV, even if I have to use a PSeye, i think we can track just the arms, and the hands positions to "correct" and fill in the blanks for the leap so it can concentrate on fingers mostly. So far it is just theory, so if anyone has a PSeye, and the leap, please give it a go.

I've noticed that LEAP tracks the palm position and rotation pretty robustly, even when the fingers are going mental.

It would be really cool if LEAP released an alternate tracking mode that was far more robust, and only tracked the palm position/orientation. However, right now LEAP doesn't have a far enough tracking volume (normal to the camera orientation) to be super useful for VR I think. If the average person loses tracking completely when they completely extend their arms, a robust and good UX for most people just isn't possible. :(

:idea: Instead of using external mechanics for haptics and force feedback instead have the player put on a lycra jump suit with lots of electrodes for each muscle then you can use the players own muscles to provide full force feedback! And the electrodes behind the ears can provide galvanic vestibular stimulation:


Providing simulated motion, avoiding motion sickness. Muscle stimulated force feedback would of course need to be combined with accurate body tracking sensors and markers on on the lycra jumpsuit could help with that.

Yeah I know something like that would never become mainstream but I thought it was an interesting thought.

What do you think of using GVS by itself to to provide a sense of motion.

I was expecting them to come out with something like this except make it prettier of coarse:


Those triggers and buttons look rather unergonomically positioned also.

"mptp" wrote:
I've noticed that LEAP tracks the palm position and rotation pretty robustly, even when the fingers are going mental.

It would be really cool if LEAP released an alternate tracking mode that was far more robust, and only tracked the palm position/orientation. However, right now LEAP doesn't have a far enough tracking volume (normal to the camera orientation) to be super useful for VR I think. If the average person loses tracking completely when they completely extend their arms, a robust and good UX for most people just isn't possible. :(

the trick is to go abstract - I worked on a project for the UCL in London recently where you guide flocks of birds with a Kinect. To guide the birds through a courtyard at the university your arms are not long enough, so I just abstracted the hands into glowing balls and amplified each movement ... so now you could reach into the far corners of the courtyard and guide the birds there.
I have been preaching this to Leap Motion for a long time - they will wake up when Intels RealSense (or another tech) takes over I guess. In other words when it is too late and nobody cares for it anymore. The limited tracking was easy to work around in abstract models, but next generation of tracking hardware will offer a much bigger tracking area.
Since they do not seem to plan to release a Leap 2 and instead go embedded I doubt we will see much from them in the future anyway. Too bad, it was such a nice tech.

I Heard something about "DragonFly", it is supposed to be their next Leap Controller, and designed with VR in mind from the beginning . After talking about this I almost bet our Leap guys chimes in here. I bet he will have more to say, but if not I'll look for the article if anyone is interested.