Skip to main content
Up and Coming
January 13, 2014

Should Oculus bundle motion controllers with every Rift?

Hi there!

I've mentioned this a couple times back around September, but now that it's official that Oculus VR is focusing on a seated experience and that external optical tracking looks likely for CV1 i think it's become relevant again..

Also posted a comment on Reddit the other day, just thought i would create a new thread here to see if we can get a bit more feedback.
http://www.reddit.com/r/oculus/comments/1uw0nl/the_next_big_hurdle_for_vr_getting_your_hands_in

Here's a slightly edited version of my reply to the OP:
"PatimPatam" wrote:

Basically, i believe that the company in the best position to put our hands inside the game is actually Oculus VR itself.

I would love to have an affordable haptic feedback glove, but for the 1st generation of consumer VR i don't think we really need one; a motion controller similar to the PS-Move but where each finger rests in a separate analog button (with optional force feedback) should be enough to simulate our hands. For instance I can imagine it having the shape of a prism or a dumbbell, with several LEDs on both ends.

Having already an external camera for positional tracking this type of controller should be very cheap to produce for Oculus (basically just plastic, a few LEDs, a bluetooth transmitter, and one of the awesome IMUs they already developed to use inside the headset).

I think Oculus could perfectly bundle 2 of these controllers with every Rift sold, without adding more than $40 or $50 to the final price.

I know it may not sound like super-exciting high-tech at first, but the fact that EVERY Rift user would have the same VR input would make a MASSIVE difference. Developers could create VR games and applications with very intuitive interfaces from day 1 knowing that there is a standard input shared by all users.


And here's a very rough sketch i made to have an idea of what i mean. On top of that there should probably be a strap that goes around the back of your hand, so the controller stays in place even with the palm open.



Also optionally there could be a thumbstick instead of the thumb button, but i don't really think it's necessary since i believe avatar movement/rotation could be archived in more intuitive ways, like pressing a button and using positional tracking (lean forward to move forward for instance). Thumbsticks for movement have become quite standard between gamers but i don't think they are that intuitive for the average person.

More on that here:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=18694


Obviously the main technical difficulty with this solution would be how to differentiate controller LEDs from HMD LEDs. The more trivial approach would probably be using a visible spectrum camera instead of infrared and have different colors for each controller but that has it's own set of disadvantages.. anyway i think with the team of computer vision experts at Oculus this should be certainly doable.

I'm also quite certain that a system like this (if used at an optimal distance from the camera) could be much more accurate than alternatives like magnetic tracking (mainly due to the magnetic distortion problem).

At the same time systems like the STEM or PrioVR could still be useful for scenarios where the user wants to move around with more freedom within a confined space, or inside omni threadmills.


I've seen quite a few people complaining about the external camera solution.. Am i the only one INSANELY excited about it?? I believe this could potentially bring us one step closer to the Oasis console.

I honestly think this should be a no-brainer for Oculus, and actually i'm betting they already have some hand-controller internal prototypes; then again maybe i'm wrong :-)

Thoughts?


EDIT:

Like i mentioned later in this thread, if occlusion is too troublesome to sort out with only the existing front camera, a 2nd inside-out camera with a wide FOV could be used on the headset to help track the controllers. If the HMD is blocked by the controllers, the relative position between the external camera and the HMD could be calculated integrating the relative position between camera and controllers and the relative position between controllers and HMD.

Maybe this inside-out camera could be used at the same time for pass-through functionality similar to Gear VR. Obviously this would add both weight and cost to the HMD, but i don't see it as a show-stopper.

    132 replies

    Up and Coming
    January 29, 2014
    "mrgreen72" wrote:
    But for the sake of discussion, let's assume we're talking about flawless hand controls. ;)


    Well hell, it would be totally badass then.
    Brevis / DevBlog Twitter
    Up and Coming
    January 29, 2014
    Tracking hand position is probably a simple task in the grand scheme of things. Tracking fingers and hand manipulation is probably way up there in being an achievable consumer friendly product. Haptic feedback would be the peek until direct CNS interaction can occur. For the beginning, the equivalent of a 3D mouse input seems adequate to my "good enough for government work" mindset. Then again, I have limited vision, and perhaps Oculus is thinking bigger picture when it comes to VR interface.

    Out of interest, since VR has been thriving in the industrial and militar area for the last 20 years, what have they come up with that is the best solution so far? From the little I've seen, there's been almost no use of a mouse and keyboard. Even those areas use some sort of 3D controller or a large control interface.
    Inspiring
    January 30, 2014
    Short version :
    - Instead of having non PC Gamers pay more for the Oculus Rift : By bundling game controllers with it.
    - Why not instead have them pay less : By including the position tracking (which may include a camera), the PC cabling, some default 3D game controllers and whatever else the non PC gamers won't be needing, in a separate PC Gaming Bundle (or the Playstation/XBox/Apple Gaming Bundle version) ?

    This would result in a less then 250$ single Oculus Rift model sold on it's own (just the HMD in a small case with a USB cable), and a single Oculus PC Gaming bundle that includes everything else the Crystal Cove case contains (plus a pair of cheap wired 3D controllers) except for the HMD itself. Gaming bundles for closed systems (XBox, PS, Apple...) can come later on, as a replacement for the initial PC Gaming bundle (while still using the same single model of Oculus Rift).

    I understand it would be better if there was a single Gaming Bundle for all the platforms, but those closed platforms don't want any open hardware to connect directly to their platform (Microsoft doesn't even let you connect your own AUDIO headset directly to their XBOne ! ), and letting those greedy companies sell their own platform specific Gaming Bundle for the (single) Oculus Rift would solve that issue.

    But I think the real issue about 3D controllers is the pricing : If nobody plans to produce cheap wired 3D controllers (meaning less then 50$) in time for the Oculus Rift launch, then it won't matter if Sixense is selling some wireless ones for over 200$, because no commercial game will be able to depend on their presence for their game development (so the lot of their games will be fully designed to comply with the heavy 2D Gamepad limitations instead).

    ----------------------------------------
    Long version :

    You could do the opposite of bundling more things :
    1) A self contained Oculus Rift with a built-in USB connector that can run limited 3D demos on it's own, using a USB gamepad or even no entry at all (selecting options just by looking at them). The minimal requirement would be a virtual Cinema room to watch 3D films in. Essentially you're buying a giant 3D TV emulator with fixed point headtracking enabled (but no position tracking available or required for it).
    2) A separate gaming bundle that adds the Oculus Camera (for position tracking and maybe also controller tracking), a default pair of cheap (wired) 3D controllers, and all the cabling required to connect that Oculus Rift to a PC. The more you bundle here, the cheaper the non PC gaming Virtual Cinema Oculus Rift entry point becomes.

    This way you get :
    - An even cheaper Oculus Rift entry price (or a better display screen for it). Good enough for watching 3D films by itself (with no PC or cables required to watch them).
    - The separate PC gaming bundle to add positional tracking and link the display to a PC, that do include a pair of cheap (wired) default 3D game controllers so every game developer can safely depend on it.
    - The ability for Oculus to update the PC gaming bundle separately from the Oculus Rift.
    - The ability for partners like Sixense and Omni to replace the Oculus Gaming Bundle with their own more expensive 360° (wireless) position tracking and 3D wireless controllers, and maybe even their own full wireless HDMI connection to the PC.
    - The ability for competitors like Microsoft, Sony, Nintendo, Apple and more to provide their own self dedicated Gaming Bundle for the Oculus Rift instead of having to compete against it. With Oculus just having to develop a single Oculus Rift for all of them (and those competitors grabbing their profits from overpricing their self dedicated Gaming Bundle for it, instead of wasting their resources on developing and overpricing their own HMD).
    - The ability for HMD competitors like Sony to develop their own HMD that is compatible with the Oculus Gaming Bundle (making it instantly compatible with all the Oculus Rift designed games, which everyone would profit from).

    There might even be more customers wanting to buy a dead simple 200$ Movie Viewing Oculus Rift (including buying it as a gift for someone), then customers wanting to buy a more complicated and slightly frightening 300$ PC gaming Oculus Rift. And once they get their hands on a 3D Movie Viewing Oculus Rift, odds are many of them will want to take the extra step by buying a (100$) gaming bundle for it (but not necessarily the one for the PC).

    And even if over half the Oculus Rift owners end up not buying a Gaming Bundle for it and are happy just using it to watch 3D films in a virtual Cinema (in a unique way and for much cheaper then any other film viewing dedicated HMD out there) : Oculus still sold over twice the amount amount of Oculus Rift HMDs, which is the right way to reduce the production cost per unit through bulk orders (maybe even securing itself a dedicated display screen for the second consumer version).

    Truth is, the first time I heard about the Oculus Rift (in 2012), I wasn't searching video game stores for a revolutionary Virtual gaming HMD : I was searching the TV stores for a Virtual TV HMD (and landed on a cheaper and more promising Oculus Rift, if only because none of the virtual 3D TV HMDs from 2012 was worth buying no matter how expensive they were).

    Not to mention how easy it would be for a supermarket to promote the Oculus Rift separately from it's PC gaming bundle as a 200$ self contained Virtual Cinema HMD (alongside the already existing more expensive Virtual TV HMDs they already have for sale), compared to how difficult it would be for the same supermarket to promote the Oculus Rift as a one of a kind 300$ PC gaming peripheral ("no sir, you can't hook it to your console, plasma TV or even Apple Computer. Only to a beefy PC, and it's not an option it's a requirement"). Basically I'm saying that Supermarket is better off selling it as a self contained Virtual Cinema HMD with zero issues, then dealing with the many PC related issues of it being sold together with the Gaming Bundle as a PC gaming peripheral (including dealing with the motion sickness issues from running that gaming Oculus Rift on a crap PC with no discrete GPU that was purchased from that same supermarket a year ago... which will show it's gaming limitations 10 times over on an Oculus Rift).

    So I came with the opposite conclusion to yours : Focus on selling more Oculus Rift (screens) by marketing it as a cheap as dirt Virtual Cinema to watch 3D films on, and make the PC Gaming Bundle (including position tracking accessories, 3D gaming controllers, PC HDMI cabling...) a complete but separate purchase. Because selling the most possible Oculus Rifts comes first, and giving the opportunity for commercial games to fully depend on default 3D controllers for their game development comes second, and this way of doing things gets both of these done.
    Up and Coming
    January 30, 2014
    @Cyril

    There are a bunch of reasons why I don't agree with you here:

    - I rather doubt the lack of positional tracking would allow OVR to sell the Rift for $100 less. The BOM for that camera and LED's must be less than $20.

    - Positional tracking is A MUST! The number of application in which positional tracking is not required are limited. And you can count that people who buy the Rift, no matter what version will want to engage in a few VR experiences where PT will be extremely useful either for interaction purposes or to avoid sim sickness.

    - Even in experiences like Cinema VR, you still need to interface with the virtual world and menus. An invisible keyboard, an invisible mouse and an x-box controller are ill suited controls for a HMD.

    - Being that the Rift will already have the camera and positional tracking, I speculate that adding an optical hand control (such as the Move) would not add much to the price of the rift. Maybe $50, or even less.

    - Allowing content creators to count on a hand device, would greatly enrich the VR experience across the board, allowing a much more natural (and less superhuman) method of interaction with the virtual world, becoming way more immersive overall, and allowing people to always see their hands and arms. If content creators know that only a small part of their users have a hand control, they will not bother doing that.
    Up and Coming
    January 30, 2014
    It's not a bad idea Cyril, but another problem on top of what alejux mentioned is that, from a marketing perspective, i don't see Oculus selling different versions of the Rift at this early stage.
    Up and Coming
    January 30, 2014
    I think segregating the product is good but doing it that much would be more detrimental than good. OVR needs to ship a single package that is excellent, functional, dependable, scaleable, etc... And removing core elements like positional tracking from the root product would be a bad idea. Developers could only absolutely depend on the bare minimum product and very few people would be interested in removing such core elements from the package since it certainly wouldn't get it under $100 - thats definitely not happening.

    Manf, shipping and stock is a concern too. Moar packages for core products that should be a single one at launch would only increase deployment costs.
    Brevis / DevBlog Twitter
    Up and Coming
    January 30, 2014
    Ok reposting my last reply to nateight in Reddit here, since i'm pretty sure no-one is going to read it by now..

    I get a bit more technical in there about my solution, in case someone is interested.

    "nateight" wrote:
    We don't even fully disagree; you're absolutely correct that this unified head-and-hand system would be "simpler", I just don't see even your excellent barbell controller suggestion being something "simple" to deal with when the camera is necessarily external. The algorithms would need to deal with sudden occlusion of the whole HMD, multiple marker sets appearing in arbitrary planes, and I still don't see a good way to deal with a satisfying range of arm movement with only one camera involved. Bringing multiple cameras into the equation is a non-starter considering the semi-Luddite consumer target market. The game Oculus seems to be playing right now is to acknowledge hand tracking is super important, but unifying everything is so challenging they aren't even going to lead the way until they get the basic Rift experience nailed down.

    Head-and-hand via optical markers makes a lot more sense to me when the markers are external, but this runs into an even worse version of the problem of expecting consumers to measure out a precise capture space. I have an idea that is so crazy it could potentially solve all these issues, but even if I can get the algorithms working I get the sense it's not something home users would be entirely comfortable with.


    "PatimPatam" wrote:

    Actually i did think a bit about these problems.

    I agree that a 2-camera solution is a no-go. But i think having a wide FOV camera and a sensible distance between the camera and the user could cover most cases. I know it's not ideal for computer-desk use, but i think users would understand that they simply need to back off a bit in order to be in a good tracking range, and the benefits would be more than worth it.

    If you look closely at the pattern in front of the Crystal Cove prototype you can notice that it's clearly built to deal with hand occlusion (in a similar way that the Posittron was), that's why they used so many LEDs. This was more or less confirmed by Brantlew in a reply to one of my questions, here:
    http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=16072&start=100

    If you take my dumbell design and think about the possible hand positions, you will notice that it's actually quite hard to have one completely hiding both ends of the other one. In those rare instances you could just rely on the IMU and assume that one controller is behind the other one.


    Once the problem of HMD occlusion by arms/hands is out of the way, the main issue left is just how to reliably distinguish the LEDs from each controller and from the HMD (3 different types).

    The more naive approach i can think of is going back to visible light LEDs and camera: when i was testing the Posittron thingy i was able to reliably recognize 6 different colors in different light conditions, using pretty much random LEDs and the only cheap camera i had available.

    Also from the top of my head i can see a few other options, like using different patterns, IR LEDs with different wavelengths, or since hand tracking doesn’t require as small latency as head tracking even pulsating LEDs, or a combination of all that.


    In the end i'm not saying it's trivial problem to solve, but i believe it's certainly doable; only this time i don't fancy spending 4 months of my free time proving it, just for the fun of it; been there, done that :-P
    Inspiring
    January 30, 2014
    Your answer suggests a few misunderstanding :

    Check the content of the full Oculus Rift case : I'm moving everything from it (all the PC related cables, the camera, the HMDI/USB adapter box, the 50$ Virtual Glove you want to add, the software, most of the documentation, maybe even the additional optic lenses...) to a separate gaming bundle, including the huge case itself really (the Oculus Rift case would only be slightly bigger then the Oculus Rift itself, with a USB cable and a sheet explaining how to view a film). That's what I value as removing 50$ of hardware from the current Oculus Rift, instead of adding 50$ of hardware to it (by including Virtual gloves with it), resulting in a rough 100$ price difference (250$ instead of 350$).

    And because that Gaming Bundle would be required to unlock direct access to PC software, 100% of PC users would necessarily own whatever 3D controller is included with that separate Gaming Bundle. Meaning content creators COULD still count on every last PC user owning that 3D controller.

    Also means you could buy the second Consumer Oculus Rift without having to buy back a second set of all it's PC gaming accessories (including not buying a second pair of 3D controllers).

    PS : The PC Gaming Bundle would not include the Oculus Rift itself, there's would still only be one single model of Oculus Rift for sale, meaning that single model would necessary include any Leds (even if you need the PC Gaming Bundle to start using them for positional tracking). Saying you need a 3D controller and position tracking to use an HMD as a virtual cinema, and that a virtual cinema can't sell a HMD on it's own merit, is denying that those virtual cinema headsets from other companies are already being sold (and neither require a full 3D controller or positional tracking to exist).
    Inspiring
    January 30, 2014
    The more I think of it the more I like the idea of banking on the tech already developed for the head to track the hands. Well, assuming they keep that solution anyway... But it looks like it.

    I wouldn't have the first clue on how to code this myself but since they seem to be doing such a good job tracking the head, and assuming its possible to have multicolored infrared LEDs easily differentiable in various lighting conditions, it seems like they're more than half way there.

    Like you said, I don't think occlusion should be that much of a problem. Well, maybe it would ruin VR nose-picking simulators... ;)

    I don't know how CPU intensive tracking those LEDs is? Of course it must not add latency to the head tracking. I guess you could track the hands in a different thread?

    Does anyone know where the tracking camera is plugged by the way? On the PC or did they put a brain in the control box?
    Up and Coming
    January 30, 2014
    "Cyril" wrote:
    Your answer suggests a few misunderstanding :

    Check the content of the full Oculus Rift case : I'm moving everything from it (all the PC related cables, the camera, the HMDI/USB adapter box, the 50$ Virtual Glove you want to add, the software, most of the documentation, maybe even the additional optic lenses...) to a separate gaming bundle, including the huge case itself really (the Oculus Rift case would only be slightly bigger then the Oculus Rift itself, with a USB cable and a sheet explaining how to view a film). That's what I value as removing 50$ of hardware from the current Oculus Rift, instead of adding 50$ of hardware to it (by including Virtual gloves with it), resulting in a rough 100$ price difference (250$ instead of 350$).

    And because that Gaming Bundle would be required to unlock direct access to PC software, 100% of PC users would necessarily own whatever 3D controller is included with that separate Gaming Bundle. Meaning content creators COULD still count on every last PC user owning that 3D controller.

    Also means you could buy the second Consumer Oculus Rift without having to buy back a second set of all it's PC gaming accessories (including not buying a second pair of 3D controllers).

    PS : The PC Gaming Bundle would not include the Oculus Rift itself, there's would still only be one single model of Oculus Rift for sale, meaning that single model would necessary include any Leds (even if you need the PC Gaming Bundle to start using them for positional tracking). Saying you need a 3D controller and position tracking to use an HMD as a virtual cinema, and that a virtual cinema can't sell a HMD on it's own merit, is denying that those virtual cinema headsets from other companies are already being sold (and neither require a full 3D controller or positional tracking to exist).

    Personally if i was Oculus i wouldn't try to compete with other companies like Avegant, vrAse, etc in the portable-connect-to-Android kind of thing, and concentrate fully on PC-gaming/apps. I understand their interest in Android on the long term, but not so sure they should jump into it for CV1.

    But if that's what they want to do, and they assume the risk of users getting sick without positional tracking (which these other companies are doing), then i guess your option would make sense. Still think it would be quite complex to handle from a marketing perspective though.