Is oculus having difficulty thinking of an input device?

As far as I know this is why Oculus bought out Carbon Design Group a few months ago who developed my personal favourite controller, the xbox 360 gamepad. Apparently they were already working with them for over a year before the buyout so I still have high hopes they are working on something magical but I'm starting to think they may release it after the headset but with full compatibility. I personally hope they come up with a glove or hand recognition camera on the rift. I have a hydra and while it feels great to move your hands around it doesn't feel very natural holding a controller and pressing a button to grab something.

You're damn right Oculus is having difficulty coming up with an input device. They've stated the reasons themselves, so I'll just restate what Oculus themselves have said several times:

Oculus aren't going to compromise on Input - if an input device ships with CV1, it's basically going to become the industry standard for VR input for many years.
The reason for this: imagine everybody with a Rift has some control device. After that, it's fair to say that more or less every VR game will have to implement that control device. Over years, developers become comfortable working with this control device and are hesitant to change. Large studios are unwilling to take risks on less-supported input devices, even if they're superior. Even if Oculus release a new input device bundled with CV2, VR consumers will have to own the old input device if they want to play old games, and not everyone will have the new input device (since not everyone will upgrade), meaning the new input device won't be as widely supported. These things slow down VR

The only ideal input is this: a skeletal hand tracker that has sub-millimetre accuracy, very good latency, and 100% reliability.
This is what Oculus have said their input device needs to be. There are a two (very big) problems with this goal however:

• If both hands are being tracked, this means that you can't hold onto a controller (or if you do, then that hand can no longer be used for interaction). The main problem with this is locomotion: how do you move around in the game if both your hands are unavailable to move joysticks / press buttons? It's something that's talked about a great deal so I won't mention it any further.


• At the moment, all hand-tracking technologies are either too expensive to be bundled with every Rift (e.g. IMU-based tracking) or too unreliable/inaccurate (e.g. optical tracking). This basically means Oculus have to come up with a completely novel hand-tracking solution

Everyone's response to Oculus' spiel about Input (and mine) is "It doesn't matter, they should just come out with *something*!
To which I partially agree, and partially disagree.
On one hand, as a developer, knowing that every single user had some form of 3D input (e.g. hydras) would be tremendously exciting and really open up a lot of creative possibility.
On the other hand, I know as a VR evangelist that hand-tracking is without doubt the future of VR input, and I know that if Oculus bundle anything else with CV1, that basically guarantees ubiquitous hand tracking by 2-5 years, or more. And I'm willing to sit through a few more years of a fragmented input-device market if it means that every headset comes with perfect hand-tracking.

As for Carbon Design - Oculus definitely didn't buy them to design them an input device, they bought them to design CV1. Sure, having an in-house design team means that they'll also be designing CV1's camera, all the boxes, cable housings, marketing material, etc. Once Oculus have an input device all nice and working properly, Carbon Design Group will get busy making it look pretty. But designing the Input Device definitely wasn't their main mission.

"Alric" wrote:
As far as I know this is why Oculus bought out Carbon Design Group a few months ago who developed my personal favourite controller, the xbox 360 gamepad. Apparently they were already working with them for over a year before the buyout so I still have high hopes they are working on something magical but I'm starting to think they may release it after the headset but with full compatibility. I personally hope they come up with a glove or hand recognition camera on the rift. I have a hydra and while it feels great to move your hands around it doesn't feel very natural holding a controller and pressing a button to grab something.

I think having a controller will really have less impact on the industry than if it were motions. Personally I wouldn't see myself buying OR if they used some sort of controller.

"mptp" wrote:
You're damn right Oculus is having difficulty coming up with an input device. They've stated the reasons themselves, so I'll just restate what Oculus themselves have said several times:

Oculus aren't going to compromise on Input - if an input device ships with CV1, it's basically going to become the industry standard for VR input for many years.
The reason for this: imagine everybody with a Rift has some control device. After that, it's fair to say that more or less every VR game will have to implement that control device. Over years, developers become comfortable working with this control device and are hesitant to change. Large studios are unwilling to take risks on less-supported input devices, even if they're superior. Even if Oculus release a new input device bundled with CV2, VR consumers will have to own the old input device if they want to play old games, and not everyone will have the new input device (since not everyone will upgrade), meaning the new input device won't be as widely supported. These things slow down VR

The only ideal input is this: a skeletal hand tracker that has sub-millimetre accuracy, very good latency, and 100% reliability.
This is what Oculus have said their input device needs to be. There are a two (very big) problems with this goal however:

• If both hands are being tracked, this means that you can't hold onto a controller (or if you do, then that hand can no longer be used for interaction). The main problem with this is locomotion: how do you move around in the game if both your hands are unavailable to move joysticks / press buttons? It's something that's talked about a great deal so I won't mention it any further.


• At the moment, all hand-tracking technologies are either too expensive to be bundled with every Rift (e.g. IMU-based tracking) or too unreliable/inaccurate (e.g. optical tracking). This basically means Oculus have to come up with a completely novel hand-tracking solution

Everyone's response to Oculus' spiel about Input (and mine) is "It doesn't matter, they should just come out with *something*!
To which I partially agree, and partially disagree.
On one hand, as a developer, knowing that every single user had some form of 3D input (e.g. hydras) would be tremendously exciting and really open up a lot of creative possibility.
On the other hand, I know as a VR evangelist that hand-tracking is without doubt the future of VR input, and I know that if Oculus bundle anything else with CV1, that basically guarantees ubiquitous hand tracking by 2-5 years, or more. And I'm willing to sit through a few more years of a fragmented input-device market if it means that every headset comes with perfect hand-tracking.

As for Carbon Design - Oculus definitely didn't buy them to design them an input device, they bought them to design CV1. Sure, having an in-house design team means that they'll also be designing CV1's camera, all the boxes, cable housings, marketing material, etc. Once Oculus have an input device all nice and working properly, Carbon Design Group will get busy making it look pretty. But designing the Input Device definitely wasn't their main mission.

Like I said in another post, I'd be willing to spend a lot in VR controls, as long as it's done right. But whatever they choose will be the norm for many years to come unless another manufacturer (i.e. sony's Morpheus) comes up with another device.

I don't know how much this vr gloves/hand tracking devices cost, but if they cost a lot then shouldn't we wait a few more years so that the technology viable to buy.

Great post mptp !

I didn't know all of that, so you've really cleared things up for me and yes I think you are right regarding carbon, they will be the ones making everything look nice. Do you think samsung made the gear vr look consumer ready? I guess so as they are making it themselves....

I've just purchased a leap motion but not tried it yet. I did a bit of research and they have prototypes of a new version made specifically for vr. They have said though that they are not planning to release it as a product but bundled with something. Does that mean with Oculus or other hardware input companies. Time will tell.

I was also excited to see that a few of the Oculus guys have backed the Nimble Sense on kickstarter including Brendan Iribe so that maybe means they are further away than we think to finding an input solution for vr.

I really hope they do come up with something for cv1. Even if it is not perfect but the basis for the right input, which can be improved over time. A bit like the cv1, I'm pretty sure it will be 1440 or 1600 so then cv2 will be 4k etc etc.
But that goes against them not compromising on input and so unless they are having brainwaves right now to get it nailed in time then I think cv1 will be headset only.

I remember an interview with Palmer maybe a year or so ago and he was asked if they had tried an external camera on the rift and he said "Of course we have". Which obviously means they haven't overcome all the issues that come with that yet. I'm convinced vr is the future of gaming & certain entertainment but more and more it is evident that vr will become it's own category, with games and input and content made specifically for it. Everyone I mention vr to says imagine call of duty or bf4 on it but I keep telling them that vr doesn't work like that. Developers need to think outside the box for exciting vr content that doesn't make people ill. Oculus can only do so much to reduce motion sickness.

Time will tell and it's so exciting being part of this vr journey. I look for any snippet of news I can and the long talks Carmack and Abrash have done boosts by excitement to new levels. They really give us the info how good vr can be and the obstacles that need to be overcome.

The only reason I said a glove was to get some kind of haptic feeling in it. I've only used hydra but how amazing would it be to get some kind of feeling when your finger touches something in game. That must make a huge difference to just watching your hand touch something but to actually feel a kind of collision. But then how would that work up against a wall, clipping nightmare. So many obstacles, but luckily oculus have the best brains on the problem.

"mptp" wrote:
The only ideal input is this: a skeletal hand tracker that has sub-millimetre accuracy, very good latency, and 100% reliability. This is what Oculus have said their input device needs to be. There are a two (very big) problems with this goal however:

• If both hands are being tracked, this means that you can't hold onto a controller (or if you do, then that hand can no longer be used for interaction). The main problem with this is locomotion: how do you move around in the game if both your hands are unavailable to move joysticks / press buttons? It's something that's talked about a great deal so I won't mention it any further.


• At the moment, all hand-tracking technologies are either too expensive to be bundled with every Rift (e.g. IMU-based tracking) or too unreliable/inaccurate (e.g. optical tracking). This basically means Oculus have to come up with a completely novel hand-tracking solution

You make some good points but i really don't think these 2 problems you mentioned should be so hard to overcome:

• If the tracking is good enough you don't really need buttons and thumbsticks, these could be simulated by doing specific movements with your hands and fingers (we simply need to find combinations that are easy to do on purpose but hard to make by mistake).


• For me it's clear that the best technology in this case is optical tracking + IMU for general hand position (which we already have for the hmd) + mechanical tracking for the fingers. These 3 technologies combined can be cheap, accurate and reliable at the same time. As and added bonus you could even have haptic feedback or "finger movement blocking".

If you have a mic and want to pause the game, all you say is "pause" and the game pauses.

In theory. I have an xbox one and my girlfriend laughs at me constantly shouting "XBOX ON!!!" nothing...... XBOX..... XBOX RECORD.......nothing!" never ever works. All my friends joke about it too, it literally doesn't work properly.

"PatimPatam" wrote:
You make some good points but i really don't think these 2 problems you mentioned should be so hard to overcome:

• If the tracking is good enough you don't really need buttons and thumbsticks, these could be simulated by doing specific movements with your hands and fingers (we simply need to find combinations that are easy to do on purpose but hard to make by mistake).


• For me it's clear that the best technology in this case is optical tracking + IMU for general hand position (which we already have for the hmd) + mechanical tracking for the fingers. These 3 technologies combined can be cheap, accurate and reliable at the same time. As and added bonus you could even have haptic feedback or "finger movement blocking".

Yeah that's true - perfect skeletal hand-tracking is basically an infinitude of buttons and joysticks. But the reason you still need a seperate locomotion controller is you need to have the ability to use both your hands while you move around. If one of the hands is constantly being used for locomotion, then you might as well just give the player a nunchuk or something. (Obviously a hand is better than a nunchuk since you can 'dismiss' and 'recall' the controller at will, but you get my point?)

You don't need mechanical tracking at all - if you know the hand's root position (through optical/magnetic tracking), and you have IMUs on each finger bone, then you can apply a hand-model to get a skeleton reconstruction. This works because the finger IMUs are only doing rotation detection, not acceleration. Unfortunately, it's not a cheap solution at all - an optically tracked data glove with 6 IMUs in it (which is the bare minimum, ideally you'd want 15, but 11 would do a very good job) would cost consumers anywhere between $50 and $100 to buy, and you need two of them. Even using a mechanical solution like Dexmo wouldn't get the cost much lower (and mechanical data gloves have their own problems)
I remember reading that the 'sweet spot' for consumer electronics was around $200-$300. That is, if you go much higher than that, you start losing big chunks of adopters. It's looking like CV1 is going to perhaps just scrape in at around $300 - adding $50-$100 to the price for data gloves is a heavy decision to make.

thanks for that