Walking in VR [Long, Long, Long Post]

[mptp]

I keep finding myself posting really long posts in random threads that start talking about locomotion in VR, so I'm going to make a big post here instead that I can just point to.

Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.

The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:

• The Virtuix Omni
  The first of the harness-style omnidirectional treadmills. A curved, low-friction surface, special shoes, and a weight-removing harness allow the user to convert the natural walking and running motions they use everyday into virtual movement commands.


• The Cyberith Virtualizer
  Following the Omni's example, the Virtualizer works similarly to the Omni, but allows the user to crouch and jump while remaining in the harness, allows the user to play in socks rather than special shoes, and has a flat surface rather than the Omni's curved one.


• The Walkmouse
  A large-circular panel inset with many small rollers, this treadmill simulates a moving surface over which the user can walk.


• The StinkyBoard
  Effectively a four-button gamepad for a single foot, this device allows the user to emulate WASD keyboard events using the feet, and so achieve simple VR locomotion


• The Wii BalanceBoard
  Not actually a PC peripheral, with some hacking can be used as a lean-controller, allowing the user to move in VR by leaning their body forward, backwards, left and right


• Stompz
  Implements IMUs on each feet to allow the user to make 'stomping' gestures to move forward, left and right in game


• Infinideck
  A more traditional-style treadmill that can move in any 2D direction, that the user walks on as they would walk on a normal surface. Unfortunately, must be manually controller by a second person at the time of this writing


• WizDish
  A low-friction surface that removes the vertical foot-motion component of walking, leaving the user to make skii-ing like gestures to control motion. At the time of this writing, lacks community support and/or funding


• Free-Walking
  Perhaps the most immersive of them all - the user walks around a physically-large space (like a football field or warehouse), with real walked distance corresponding 1:1 to virtual walked distance


• LocoVR
  A crowd-sourced project involving what is described as its conceiver as "basically a cross between a small bar stool and a kind of oversized joystick", able to rotate freely and respond to lean from the user to provide locomotion input


• Pointman
  In use by the US Navy for infantry simulation, the Pointman system is actually a combination of several input devices. One of them is a device comprised of two footpedals able to move forward and backwards, as well as to be pressed down like a car acceleration pedal. At the time of this writing, a handheld controller is required for turning.

Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.

• Price
  This one is a bit of a no-brainer. The cheaper something is, the more people will buy it. I would say that once the price of a device goes above about $150, it becomes a serious barrier to widespread adoption. Not that the ideal VR walker should necessarily be cheaper than this, but it's a good ballpark figure to go for.


• Size
  Most people live in cities, meaning most people have very limited living space. That is to say, people tend to be less than impressed if asked to reserve an area the size of a double-bed for a VR input device. So the physical size of the device will be a further barrier to widespread adoption. Ideally, the system will fit onto the user, so it effectively takes up no space in and of itself.


• Flexibility
  Keep in mind that locomotion in games isn't always human-style walking/running. For example, think of Minecraft with a Virtuix Omni. The Omni would work great while you're running around mining, building and fighting zombies. But if you want to play in creative-mode, suddenly you have this big mismatch between flying around in-game, but making walking and running motions in reality to drive this motion. For a device to become widely-used, it needs to be suited to a wide variety of locomotion applications, not just walking and running

Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?

• Movement similarity
  We all know what walking feels like. We know what the movements are, and how it feels when we perform them. The closer a VR Walker can get to emulating this motion, the more immersive it will be. In this way, devices like the Omni or the Virtualizer are a step-up from solutions like the StinkyBoard or Stompz. The obvious ideal is devices like the WalkMouse or InfiniDeck, the question of whether these solutions are currently realistic remains to be seen.


• Constraints on movement
  An important thing to consider is the degree to which a VR Walker inhibits our ability to perform actions unrelated to locomotion. For example, the Omni doesn't allow users to bend down, because they're stuck in a harness that has a fixed height off the ground. Both the Omni and the Virtualizer don't allow the user to lower their hands below their waist, since the harness gets in the way. These kinds of movement constraints are non-issues in some games, especially FPS, where the user always has their hands above their waist holding a gun, but significantly annoying in games involving peaceful exploration, where the user is likely to want their hands to rest in their natural position - down by their sides.


• Strenuousness (Spellcheck agrees with me that this is a real word)
  When we're talking about using natural motion to run around in games, obviously an issue is going to be people getting tired! I personally can't run for much more than 5 minutes before I'm completely wasted. I'm definitely not up to running around for a half-hour gaming session. This isn't only about physical capability - if people are getting tired and cramped while playing, this is an immersion-breaking experience and something to be avoided.

Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:

• Move the ground under the user
  This is the solution that WalkMouse and InfiniDeck are pursuing. This is also the solution that traditional exercise treadmills implement. All you have to do is have a device that the user stands on, and when they move, detect the speed of movement, and move the surface they're standing on in the opposite direction at that speed, negating any physical movement the user might undergo, saving them from crashing into a wall and breaking their precious precious DK2.


• Remove the frictional force
  This is the solution that the Omni and the Virtualizer are using, and many hypothetical input devices involving suspending the user from harnesses on the ceiling etc. are also operating under this principle.
  This solution typically involves both reducing the normal force (that is, taking the user's weight off the feet so that the feet are effectively suspended), and reducing the coefficient of friction between the feet and the ground (eg. socks on a polished wood floor) so that the feet can easily slide over the surface.

There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:

• Moving the ground under the user requires many moving parts, increasing cost and reducing reliability


• Ensuring that the dynamic surface behaves exactly as the user would expect a static surface to behave is 100% essential to avoid loss of balance leading to injury, and requires zero latency, which is an engineering challenge not likely to be solved any time soon

So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.

Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about

• IMUs (Inertial Measurement Units)
  Specifically, I'm talking about IMUs containing an accelerometer, a gyroscope and a magnetometer. Through sensor-fusion these three devices allow us to get drift-free rotational measurements at about 1kHz. In addition, with the advent of MEMS technology, they're pretty cheap, and digital MEMS IMUs do much of the processing on the chip, so they're easy to use.


• Optical tracking
  Optical tracking is awesome because it gives us 1:1 positional data in realtime, and this data will never drift. The main Achilles' Heel of optical tracking is occlusion - any objects between the camera and the tracked objects will interfere with tracking.


• Pressure sensors
  These devices measure pressure applied downwards on them. Four of these are all the sensors that the Wii BalanceBoard has behind it. They're quite versatile, and can be small enough to be used in wearable devices.


• Capacitative sensors
  These are effectively touch-sensors, that when placed in an array, allow the position and shape of an object in contact with the sensor surface to be resolved. These are the sensors behind the Omni.

The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:

What device removes the weight of the user off the ground so that they can use their normal walking/running motions, but doesn't impose any other movement restrictions? Does this device take up little room, and is it cheaper than $200? Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?

I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.

What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.

[brantlew]

You forgot a couple. The Wizdish is another simple slider that failed to generate much investment, and the Virtuasphere which while a bit impractical is actually available I believe.

And then there is a whole class of solutions that you missed which are the redirected-walking systems. Almost everybody dismisses them out-of-hand because of the space requirements, but I've tried 3 of these ODT devices (I won't say which) along with a primitive walking system and in my extremely biased opinion the immersion level of redirected walking is 10x as immersive as the ODT solutions. It's also the least restrictive and cheapest of the bunch.

[sutekiB]

Great post! Very thorough examination of the topic. I would say that adoption will be driven mainly by price versus performance (value for money). For me that is best provided by a passive ODT like the Omni. There seems to be a diminishing return when you start to make them more complicated and expensive with motorized or otherwise moving parts. In fact, to me those are actually worse. A couple of problems they have are inertia and response. For example, if you try to stop in a 'hamster-ball' type device, it has its own momentum which keeps it turning, so it's hard to keep your balance - I've even seen someone fall over in one.

The Omni lets you go from rest to sprinting instantly; stop, change direction and sprint again. I don't see that motorized treadmills can let you do that. How would they keep up and would it be possible to stay on them?

I don't think enough people have sufficiently large indoor spaces for redirected walking to be an option (let alone running). I can't take more than three steps here without bumping into something! Though I'd like to try it. I do see wide open spaces being used for augmented reality - I can imagine kids playing new AR sports in gyms, playgrounds and training fields.

In homes though, I think passive treadmills like the Omni will become hugely popular in the next few years. Not just for the added sense of presence, and mitigation of simulator sickness, but for the fitness benefits also.

There may be dedicated game rooms, while in others that lack an ODT, devices like stompz and balance boards are used.

Flexibility - I see this as something that may be improved on as part of an upgrade, rather than replacing the entire device. We may see continued innovation in this area, with ideas coming from passionate and knowledgeable people in the community, as OP suggests.

[lmaceleighton]

As part of an ongoing community project my small company is working on an ODT solution similar to the Omni and the Visualizer but does require a base platform and can be used on ANY carpet, and if you don't have one, an area rug will suffice. We are mainly working on a FPS Gun controller but having an ODT solution was feeling like to much fun to go without. So in building one ourselves, and as part of the Kickstarter campaign for the controller, we will be releases the plans for a great DIY rig that if put together properly should be VERY comparable to the Visualizer and Omni. I hope to have have all of the community support for this effort, as we are doing this for the community so we don't have to pay over $1000 for an ODT solution that doesn't require special shoes;O)

[latearrival]

I believe it's early days for this product, but there's also the VR Walker http://www.vrwalkerproject.com/

[mptp]

"brantlew" wrote:
...

Oh I hadn't seen the Wizdish before. It looks like it would have been pretty cool. But how does it differentiate between walking backwards and walking forwards? I also couldn't find any information about what sensors it uses, other than an unverified mention of IR depth-sensors. I'll add it anyway.
I'll not add the Virtuasphere, just because it's definitely not a consumer device. When you have to email the company to get a quote, it's not going to be accessible to the average consumer. 😉
And yeah, I agree that using a large area and simply detecting real motion and interpreting it as virtual motion is the ideal solution. However, it defeats one of the largest advantages of VR - that is, it frees us from many of the restrictions of the material world! Plus not everyone has a square kilometre of flat ground in their backyard. I'll put it in as well though, for completeness' sake. 😉
You might want to check out http://zerolatencymedia.com/ - it's a virtual reality setup in Melbourne (there are probably similar things elsewhere in the world) that uses this kind of 1:1 walking input for locomotion.

"sutekiB" wrote:
...

See for me, a passive ODT ala Omni is already too expensive, and the movement-limitation imposed by the harness isn't acceptable. I would be able to look past both of these though, since the expense will go down as more and more competition arrives, but the harness is a completely necessary component of these ODTs. I think they're a good start, since if nothing else they'll make sure everyone's keenly aware of the necessity for a good VR Walker.
I do agree that they're head-and-shoulders above any motorised locomotion input device, since the momentum is always going to be a huge problem. Even with predictive algorithms, there are always going to be relatively common use-cases where there is a sudden unpredictable change in the user's direction (e.g. going from sprinting to a sudden stop) that will be impossible for any system to keep up with.

"lmaceleighton" wrote:
...

That sounds interesting - if the surface the user is walking on is just carpet, how are you capturing the walking motion?
And more importantly for me, how are you removing the user's weight from the surface? Are you planning on using a harness like the Omni and Virtualizer?

"latearrival" wrote:
...

Looking over their website, it seems to me that they have big plans, but no details for how to make them come to fruition. I've been thinking myself about having a locomotion device that sticks on your feet, kind of like Stompz. However I lost confidence in project VR Walker (I wish they hadn't stolen my name D:) when I read this:
"It's motorized and controlled with an array of sensors. The sensors tell the shoe modules what to do to keep you from travelling through space as you go through the natural walking motion."
That implies to me that it will be a system kind of similar to this http://www.gizmag.com/go/5811/, but aimed at the consumer VR market. The problem with this is the same problem that was mentioned by sutekiB - it just doesn't seem possible that any motorized device like this will be able to track and respond to movement fast enough to keep up with sudden changes in speed or direction. Plus the vagueness of the paragraph and the lack of any further information, images or videos on the site hint to me that the project is just an idea with no testing, prototypes or money behind it.
I'd love to be proven wrong though. 😄

[Fredz]

"mptp" wrote:
If I miss any, let me know and I'll add it asap.

Older projects :
http://vrwiki.wikispaces.com/Locomotion+devices

Current projects :
http://vrwiki.wikispaces.com/accessories#Locomotion%20devices

[mptp]

"Fredz" wrote:

"mptp" wrote:
If I miss any, let me know and I'll add it asap.

Older projects :
http://vrwiki.wikispaces.com/Locomotion+devices

Current projects :
http://vrwiki.wikispaces.com/accessories#Locomotion%20devices

I should probably point out that I'm more talking about current, feasible consumer technology, as opposed to all devices that can be used as locomotion inputs. 😛
However those are really awesome links. If it wasn't approaching 2am I'd get into them right now. As it stands, might have to wait for tomorrow. But cheers, there's lots in there that I haven't seen yet! 😄

[Fredz]

Unfortunately I don't have links for most of them anymore. I had them in my previous Wiki but didn't backport to this one and now it seems innaccessible.

[mptp]

I'm sure the internet has some information on them somewhere. 😄