Up and Coming

Should Oculus bundle motion controllers with every Rift?

132 replies
34308 views

Hi there!

I've mentioned this a couple times back around September, but now that it's official that Oculus VR is focusing on a seated experience and that external optical tracking looks likely for CV1 i think it's become relevant again..

Also posted a comment on Reddit the other day, just thought i would create a new thread here to see if we can get a bit more feedback.
http://www.reddit.com/r/oculus/comments/1uw0nl/the_next_big_hurdle_for_vr_getting_your_hands_in

Here's a slightly edited version of my reply to the OP:

"PatimPatam" wrote:

Basically, i believe that the company in the best position to put our hands inside the game is actually Oculus VR itself.

I would love to have an affordable haptic feedback glove, but for the 1st generation of consumer VR i don't think we really need one; a motion controller similar to the PS-Move but where each finger rests in a separate analog button (with optional force feedback) should be enough to simulate our hands. For instance I can imagine it having the shape of a prism or a dumbbell, with several LEDs on both ends.

Having already an external camera for positional tracking this type of controller should be very cheap to produce for Oculus (basically just plastic, a few LEDs, a bluetooth transmitter, and one of the awesome IMUs they already developed to use inside the headset).

I think Oculus could perfectly bundle 2 of these controllers with every Rift sold, without adding more than $40 or $50 to the final price.

I know it may not sound like super-exciting high-tech at first, but the fact that EVERY Rift user would have the same VR input would make a MASSIVE difference. Developers could create VR games and applications with very intuitive interfaces from day 1 knowing that there is a standard input shared by all users.

And here's a very rough sketch i made to have an idea of what i mean. On top of that there should probably be a strap that goes around the back of your hand, so the controller stays in place even with the palm open.

Also optionally there could be a thumbstick instead of the thumb button, but i don't really think it's necessary since i believe avatar movement/rotation could be archived in more intuitive ways, like pressing a button and using positional tracking (lean forward to move forward for instance). Thumbsticks for movement have become quite standard between gamers but i don't think they are that intuitive for the average person.

More on that here:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=18694

Obviously the main technical difficulty with this solution would be how to differentiate controller LEDs from HMD LEDs. The more trivial approach would probably be using a visible spectrum camera instead of infrared and have different colors for each controller but that has it's own set of disadvantages.. anyway i think with the team of computer vision experts at Oculus this should be certainly doable.

I'm also quite certain that a system like this (if used at an optimal distance from the camera) could be much more accurate than alternatives like magnetic tracking (mainly due to the magnetic distortion problem).

At the same time systems like the STEM or PrioVR could still be useful for scenarios where the user wants to move around with more freedom within a confined space, or inside omni threadmills.

I've seen quite a few people complaining about the external camera solution.. Am i the only one INSANELY excited about it?? I believe this could potentially bring us one step closer to the Oasis console.

I honestly think this should be a no-brainer for Oculus, and actually i'm betting they already have some hand-controller internal prototypes; then again maybe i'm wrong :-)

Thoughts?

EDIT:

Like i mentioned later in this thread, if occlusion is too troublesome to sort out with only the existing front camera, a 2nd inside-out camera with a wide FOV could be used on the headset to help track the controllers. If the HMD is blocked by the controllers, the relative position between the external camera and the HMD could be calculated integrating the relative position between camera and controllers and the relative position between controllers and HMD.

Maybe this inside-out camera could be used at the same time for pass-through functionality similar to Gear VR. Obviously this would add both weight and cost to the HMD, but i don't see it as a show-stopper.

Show previous replies

Nukemarine

Up and Coming

"gravelbeast" wrote:
Haptic gloves - I just don't see how this is better than a Hydra setup? why do you need to control each finger? How will you walk or do other actions separate from hand movement? The basic thing you need to do with your fingers is to grab; you can do that with buttons on motion sticks.

Here's why you'd want "control" of each finger: put somebody in a room in VR without a controller and what is there reaction to objects like walls and shelves that are close to them? From what I've seen, people want to reach out and touch or interact with their hands. Although it's not likely in the near future, I think there's merit to removing the need of a obvious controller interface between the user and the virtual environment. Also as mentioned in SteamVR Dev Days, you want to put the menus and other interacting methods into the environment and not as some HUD pop-up. Being able to bring up these interfaces with just your hands will be more intuitive than button pushes on a hand held controller.

Until that time, which is quite a ways off, I like the idea of STEM or Hydra with left and right trackable controllers. I think that when that's the interface, then the hands in the game should look like they're holding something instead of the open palm position they show at the moment. Let it look like our hands are holding pincers or other tools. That'll help the virtual immersion quite a bit since we see ourselves holding something in the virtual world while we're holding something in the real world.

Tbone

Up and Coming

I brought this up in Reverend Kyle's podcast and we had a pretty good discussion about it. There were some things I hadn't thought of.

Link here (1:07:05)

One thing I found interesting towards the end of that discussion (around the 1:25:00 mark) is that Oculus may be purposely avoiding motion controllers since their positional tracking solution is currently dependent upon line of sight. If you handed someone motion controllers, they'll end up waving their hands in front of the camera, cutting off the positional tracking to the headset. I hadn't thought of that, but it makes sense. If that's the case, we won't see motion controllers widely adopted by Oculus until they find a positional tracking solution that doesn't require optical line of sight.

The Furious Angels - an Oculus Rift Online Gaming Community!

mrgreen72

Inspiring

"Tbone" wrote:
If you handed someone motion controllers, they'll end up waving their hands in front of the camera, cutting off the positional tracking to the headset.

I don't think it's that much of a problem. Like I already joked about, unless you're playing a nose picking simulator or something, chances are you won't put your hands in front of your face that much. Besides, they've put 18 infrared LEDs precisely to compensate for occlusion. I don't know what's the minimum amount the camera needs to pick up, but even if you were to block them all for a few milliseconds, the tracking would gracefully fall back to rotation which should be good enough until you get your damn hands away. :lol:

PatimPatamAuthor

Up and Coming

"Tbone" wrote:
I brought this up in Reverend Kyle's podcast and we had a pretty good discussion about it. There were some things I hadn't thought of.

Link here (1:07:05)

One thing I found interesting towards the end of that discussion (around the 1:25:00 mark) is that Oculus may be purposely avoiding motion controllers since their positional tracking solution is currently dependent upon line of sight. If you handed someone motion controllers, they'll end up waving their hands in front of the camera, cutting off the positional tracking to the headset. I hadn't thought of that, but it makes sense. If that's the case, we won't see motion controllers widely adopted by Oculus until they find a positional tracking solution that doesn't require optical line of sight.

Hey Tbone, thanks for mentioning this in the podcast and for the link; i just had a look.

I agree that occlusion is an issue, but i don't think it's a deal breaker; as long as you stay at a reasonable distance from the camera i think it's a solvable problem. I did manage to more or less overcome it with my Posittron prototype using redundancy, and that was without having an IMU (like the Rift has) in order to implement proper sensor fusion.

Please read my reply to nateight here:

"PatimPatam" wrote:
Actually i did think a bit about these problems.

I agree that a 2-camera solution is a no-go. But i think having a wide FOV camera and a sensible distance between the camera and the user could cover most cases. I know it's not ideal for computer-desk use, but i think users would understand that they simply need to back off a bit in order to be in a good tracking range, and the benefits would be more than worth it.

If you look closely at the pattern in front of the Crystal Cove prototype you can notice that it's clearly built to deal with hand occlusion (in a similar way that the Posittron was), that's why they used so many LEDs. This was more or less confirmed by Brantlew in a reply to one of my questions, here:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=16072&start=100

If you take my dumbell design and think about the possible hand positions, you will notice that it's actually quite hard to have one completely hiding both ends of the other one. In those rare instances you could just rely on the IMU and assume that one controller is behind the other one.

Once the problem of HMD occlusion by arms/hands is out of the way, the main issue left is just how to reliably distinguish the LEDs from each controller and from the HMD (3 different types).

The more naive approach i can think of is going back to visible light LEDs and camera: when i was testing the Posittron thingy i was able to reliably recognize 6 different colors in different light conditions, using pretty much random LEDs and the only cheap camera i had available.

Also from the top of my head i can see a few other options, like using different patterns, IR LEDs with different wavelengths, or since hand tracking doesn’t require as small latency as head tracking even pulsating LEDs, or a combination of all that.

In the end i'm not saying it's trivial problem to solve, but i believe it's certainly doable; only this time i don't fancy spending 4 months of my free time proving it, just for the fun of it; been there, done that :-P

Also an incorrect "fact" mentioned by Rev Kyle is that that "positional tracking is positional tracking" and they all play nicely with each other no matter the implementation.

This is really not true, even perfect absolute positional tracking always tracks position relative to something: with optical relative to the camera, with magnetic relative to the main transmitter base.

I can only see 2 options for using STEM + Oculus, none of which i really like:

1- Use user calibration input the same way Hydra works. At the beginning (and probably in the middle) of every game require the user to put their hands in a specific position and press a button. As you mentioned yourself the results are not that great, at least with the Hydra.

2- Use one of the STEM modules attached to the HMD and disable Oculus' positional tracking. Oculus claims their tracking is many times more accurate than Sixense's precisely because they can use the ultra-low latency info from the IMU to their advantage, so even if this was possible it would still provide a worse experience.

Btw i'm not the only one saying this:
http://www.reddit.com/r/oculus/comments/1w4ugk/what_does_oculuss_new_tracking_technology_mean/ceyte6q?context=3

"palmerluckey" wrote:
What you are suggesting is actually very difficult. Our tracking works as well as it does because of the tight integration between IMU and the camera, they have complementary strengths and weaknesses that have been carefully tuned. Other tracking solutions will not have that benefit, so even systems with theoretically comparable specifications will probably result in a lower quality experience. STEM is great for controllers, but magnetic tracking does not have the 1:1 absolute tracking capabilities needed for good head tracking, nor does it pair as well with an IMU.

Finally i just found a bit frustrating that no-one in the podcast apart from you seemed to even acknowledge the benefits of having an integrated system over a segregated (and more expensive) one.

Oh well, it was a good listen anyways!

PS: I do believe that Sixense is doing a great job and i will probably buy the STEM system if there is no alternative from Oculus in the end. I think having a wireless system, more precise than the Hydras and modular is a huge step forward. I simply have some concerns about the technology and about the pricing/marketing to enable broad adoption.

gravelbeast

Up and Coming

"nukemarine" wrote:
Here's why you'd want "control" of each finger: put somebody in a room in VR without a controller and what is there reaction to objects like walls and shelves that are close to them? From what I've seen, people want to reach out and touch or interact with their hands.

And they can do that with a motion stick, all a haptic glove gives you is five finger motion which seems of very limited utility compared to having discrete buttons and sticks that can be used for anything.

Although it's not likely in the near future, I think there's merit to removing the need of a obvious controller interface between the user and the virtual environment. Also as mentioned in SteamVR Dev Days, you want to put the menus and other interacting methods into the environment and not as some HUD pop-up. Being able to bring up these interfaces with just your hands will be more intuitive than button pushes on a hand held controller.

Sorry but how do you know that? Sounds sort of Kinect-y to me, as in "waving your hands and talking to your TV is more intuitive than pressing a button". They already tried something like this with the Hydra, where the standard setup makes you do weird gestures like moving your hand up to jump and down to reload. It's frankly confusing as hell.

I could see you pressing a button to bring up the menu and then navigating the menu with hand motions (just like a mouse!), but you still need that button.

Until that time, which is quite a ways off, I like the idea of STEM or Hydra with left and right trackable controllers. I think that when that's the interface, then the hands in the game should look like they're holding something instead of the open palm position they show at the moment. Let it look like our hands are holding pincers or other tools. That'll help the virtual immersion quite a bit since we see ourselves holding something in the virtual world while we're holding something in the real world.

Yup. Right now it's all about setting the standard.

Haptic gloves would be a great innovation once the standard has been set, for specific uses, much like a complex mouse. In time gloves or a similar device may even become the standard interface.

But launching the Rift with haptic gloves without any buttons or joysticks would be like releasing the Mac with a five-button mouse and no keyboard.

This of course assumes Oculus will be bundling the Rift with a controller in the first place.

As I believe this thread has made clear, it would be a big mistake of them not to.

Lately I've started to realize the implications of the fact that Virtual Reality is not a peripheral but a whole new integrated computing system.

It does not consist merely of a display unit; much like an iPad, a PC, an Xbox 360 or a Wii, it consists of a processing system, a display system, and a standard interface system.

For Oculus the processing unit (for now) is desktops and laptops, the display is the Rift and the interface is yet to be revealed. If Oculus forgets this three-legged nature of their product and launches the Rift without a standard interface, they will only have two thirds of a computing revolution. It would be like launching an Apple II when they could have launched a Mac.

(And PC is not the optimal processing unit for mass adoption. Assuming Oculus knocks the interface out of the park, I think the jankiness of the PC is the Rift's biggest Achilles' Heel and VR will be held back from fully hitting the mainstream until we get a system that works smoothly on console; I can see Sony giving Oculus its first bloody nose here. Then a few years after that VR will finally get ubiquitous when every phone can run high quality sims, and Oculus could return to unquestioned dominance, with a grizzled, battle hardened 28-year old Palmer Luckey at the helm. :D)

PatimPatamAuthor

Up and Coming

"gravelbeast" wrote:
Yup. Right now it's all about setting the standard.

Haptic gloves would be a great innovation once the standard has been set, for specific uses, much like a complex mouse. In time gloves or a similar device may even become the standard interface.

But launching the Rift with haptic gloves without any buttons or joysticks would be like releasing the Mac with a five-button mouse and no keyboard.

I don't know, if we were talking about any other company i would agree, but i think Oculus could afford to set a new standard and both developers and users would adapt to it.

If you had enough precision/accuracy and low latency, i believe anything you could do with physical buttons or joysticks could be done equally or better with a combination of finger gestures and hand motions, without being gimmicky. See my proposal for avatar movement in this same thread as an example.

"gravelbeast" wrote:

This of course assumes Oculus will be bundling the Rift with a controller in the first place.

As I believe this thread has made clear, it would be a big mistake of them not to.

Well i probably wouldn't say the word "mistake".. yes my gut feeling tells me that it would be a lost opportunity if they don't, but i want to think they will have very valid reasons if they decide NOT to bundle controllers with the Rift.

If that's the case then i still hope Oculus will release motion controls as a cheap peripheral not long after CV1 is released (before CV2). If that does not happen either then i hope that the STEM price comes down, or that a third company joins the VR input party and delivers this important missing piece.

alejux

Up and Coming

Haptic gloves are currently way, way too complex. Right now, we need something simple like the Hydra or STEM. A 1:1 stick with buttons that can track hand position and orientation. That would be more than enough for now, and it is something we could expect to see OVR bundle with the Rift at a negligible price, IMO.

redslashace

Up and Coming

Gloves won't be too hard if you keep it basic like this one by Jack Minardi.

List of Parts To make a wireless haptic feedback glove
-five modified servos(so you can read finger position alternatively add flex sensors)
-a raspberrypi
-a battery
-a bluetooth dongle
-some cableing
-a stem module / base
-some 3D printed parts
-a cheep glove
you can make a very sleek wireless haptic feedback glove. Software will be the more challenging part but not impossible.

Here are 2 part models for starters. http://www.thingiverse.com/thing:245775 http://www.thingiverse.com/thing:245771 Will design more as soon as my friends printer is fixed up.

help build the gloves of the future https://www.facebook.com/groups/536642796371093/

alejux

Up and Coming

@redslashace

It's obvious a haptics hand glove would be the ideal. But to get from that what you just showed us, to a cheap consumer level device, with individual finger sensibility, a cool and very compact design and 1:1 tracking, it will take years.

I wouldn't put my hopes on something like that coming for at least another 2 or 3 years.

redslashace

Up and Coming

I totally agree let's aim to have it perfected in 2-3 years or at least consumer ready. Oculus Rift even still isn't a perfected device. Join the cause and help make the gloves of the future a reality. If possible let's get a version one out in the wild by next year.

Show more replies

Get Help

Sign up

Login with SSO

Login to the community

Login with SSO