Walking in VR [Long, Long, Long Post]

I keep finding myself posting really long posts in random threads that start talking about locomotion in VR, so I'm going to make a big post here instead that I can just point to.

Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.

The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:

The Virtuix Omni
The first of the harness-style omnidirectional treadmills. A curved, low-friction surface, special shoes, and a weight-removing harness allow the user to convert the natural walking and running motions they use everyday into virtual movement commands.

The Cyberith Virtualizer
Following the Omni's example, the Virtualizer works similarly to the Omni, but allows the user to crouch and jump while remaining in the harness, allows the user to play in socks rather than special shoes, and has a flat surface rather than the Omni's curved one.

The Walkmouse
A large-circular panel inset with many small rollers, this treadmill simulates a moving surface over which the user can walk.

The StinkyBoard
Effectively a four-button gamepad for a single foot, this device allows the user to emulate WASD keyboard events using the feet, and so achieve simple VR locomotion

The Wii BalanceBoard
Not actually a PC peripheral, with some hacking can be used as a lean-controller, allowing the user to move in VR by leaning their body forward, backwards, left and right

Stompz
Implements IMUs on each feet to allow the user to make 'stomping' gestures to move forward, left and right in game

Infinideck
A more traditional-style treadmill that can move in any 2D direction, that the user walks on as they would walk on a normal surface. Unfortunately, must be manually controller by a second person at the time of this writing

WizDish
A low-friction surface that removes the vertical foot-motion component of walking, leaving the user to make skii-ing like gestures to control motion. At the time of this writing, lacks community support and/or funding

Free-Walking
Perhaps the most immersive of them all - the user walks around a physically-large space (like a football field or warehouse), with real walked distance corresponding 1:1 to virtual walked distance

LocoVR
A crowd-sourced project involving what is described as its conceiver as "basically a cross between a small bar stool and a kind of oversized joystick", able to rotate freely and respond to lean from the user to provide locomotion input

Pointman
In use by the US Navy for infantry simulation, the Pointman system is actually a combination of several input devices. One of them is a device comprised of two footpedals able to move forward and backwards, as well as to be pressed down like a car acceleration pedal. At the time of this writing, a handheld controller is required for turning.

Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.

Price
This one is a bit of a no-brainer. The cheaper something is, the more people will buy it. I would say that once the price of a device goes above about $150, it becomes a serious barrier to widespread adoption. Not that the ideal VR walker should necessarily be cheaper than this, but it's a good ballpark figure to go for.

Size
Most people live in cities, meaning most people have very limited living space. That is to say, people tend to be less than impressed if asked to reserve an area the size of a double-bed for a VR input device. So the physical size of the device will be a further barrier to widespread adoption. Ideally, the system will fit onto the user, so it effectively takes up no space in and of itself.

Flexibility
Keep in mind that locomotion in games isn't always human-style walking/running. For example, think of Minecraft with a Virtuix Omni. The Omni would work great while you're running around mining, building and fighting zombies. But if you want to play in creative-mode, suddenly you have this big mismatch between flying around in-game, but making walking and running motions in reality to drive this motion. For a device to become widely-used, it needs to be suited to a wide variety of locomotion applications, not just walking and running

Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?

Movement similarity
We all know what walking feels like. We know what the movements are, and how it feels when we perform them. The closer a VR Walker can get to emulating this motion, the more immersive it will be. In this way, devices like the Omni or the Virtualizer are a step-up from solutions like the StinkyBoard or Stompz. The obvious ideal is devices like the WalkMouse or InfiniDeck, the question of whether these solutions are currently realistic remains to be seen.

Constraints on movement
An important thing to consider is the degree to which a VR Walker inhibits our ability to perform actions unrelated to locomotion. For example, the Omni doesn't allow users to bend down, because they're stuck in a harness that has a fixed height off the ground. Both the Omni and the Virtualizer don't allow the user to lower their hands below their waist, since the harness gets in the way. These kinds of movement constraints are non-issues in some games, especially FPS, where the user always has their hands above their waist holding a gun, but significantly annoying in games involving peaceful exploration, where the user is likely to want their hands to rest in their natural position - down by their sides.

Strenuousness (Spellcheck agrees with me that this is a real word)
When we're talking about using natural motion to run around in games, obviously an issue is going to be people getting tired! I personally can't run for much more than 5 minutes before I'm completely wasted. I'm definitely not up to running around for a half-hour gaming session. This isn't only about physical capability - if people are getting tired and cramped while playing, this is an immersion-breaking experience and something to be avoided.

Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:

Move the ground under the user
This is the solution that WalkMouse and InfiniDeck are pursuing. This is also the solution that traditional exercise treadmills implement. All you have to do is have a device that the user stands on, and when they move, detect the speed of movement, and move the surface they're standing on in the opposite direction at that speed, negating any physical movement the user might undergo, saving them from crashing into a wall and breaking their precious precious DK2.

Remove the frictional force
This is the solution that the Omni and the Virtualizer are using, and many hypothetical input devices involving suspending the user from harnesses on the ceiling etc. are also operating under this principle.
This solution typically involves both reducing the normal force (that is, taking the user's weight off the feet so that the feet are effectively suspended), and reducing the coefficient of friction between the feet and the ground (eg. socks on a polished wood floor) so that the feet can easily slide over the surface.

There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:

Moving the ground under the user requires many moving parts, increasing cost and reducing reliability

Ensuring that the dynamic surface behaves exactly as the user would expect a static surface to behave is 100% essential to avoid loss of balance leading to injury, and requires zero latency, which is an engineering challenge not likely to be solved any time soon

So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.

Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about

IMUs (Inertial Measurement Units)
Specifically, I'm talking about IMUs containing an accelerometer, a gyroscope and a magnetometer. Through sensor-fusion these three devices allow us to get drift-free rotational measurements at about 1kHz. In addition, with the advent of MEMS technology, they're pretty cheap, and digital MEMS IMUs do much of the processing on the chip, so they're easy to use.

Optical tracking
Optical tracking is awesome because it gives us 1:1 positional data in realtime, and this data will never drift. The main Achilles' Heel of optical tracking is occlusion - any objects between the camera and the tracked objects will interfere with tracking.

Pressure sensors
These devices measure pressure applied downwards on them. Four of these are all the sensors that the Wii BalanceBoard has behind it. They're quite versatile, and can be small enough to be used in wearable devices.

Capacitative sensors
These are effectively touch-sensors, that when placed in an array, allow the position and shape of an object in contact with the sensor surface to be resolved. These are the sensors behind the Omni.

The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:

What device removes the weight of the user off the ground so that they can use their normal walking/running motions, but doesn't impose any other movement restrictions? Does this device take up little room, and is it cheaper than $200? Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?

I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.

What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.

89 REPLIES 89

Good write-up. It is certainly a challenging problem.I am currently baking on the omni, as it seems like a good tradeoff between the various factors for the moment.

You did get me thinking about ways to use it for non-walking movement. Lets take the creative mode minecraft as an example, since it is what you said, and it is a use case I hope to have. I agree that walking in the air is likely unsatisfying. However, it does have capacitive sensors, so it should be able to give foot positions anywhere on the surface. Which means you could set up some gestural controls. Say, leaning forward on one foot with the other foot back, like you are in a perpetual step, would cause you to accelerate through the air, and drawing your foot back to rest will bring you to a stop. Or perhaps, if you have something like a stem controller, you can mix in some hand gestures to fly, like extending your hand all the way up to rise (up up and away). Or maybe one controller gives you this flight control, and the other remains used for the mouse pointer, so you can interact with blocks as you fly around.

Great post and replies. I would bet there will be at least three ways to go for a walk/run in VR:

The VR walker combined with a 3D "terrain" that mimics the VR would be the ultimate amusement park

The Omni or similar input device will be for higher end systems

Kinect or similar sensor will allow standing in place and running in place to serve as inputs

Really appreciate the links and replies.

This is an amazing post. I'm going to start off by saying something that I think might come off a little negative. There's always been a stereotype with gamers being "over weight", what this will do is actually have a person move around more,
which is very, very healthy. More gamers who just sit there, much like myself at the computer, put on a lot of weight. With the extra movement in the body as a result of the Oculus DK2, this will be VERY positive. To be completely honest, I wasn't even thinking of this, only when I started to read the Wikipedia page, it just suddenly came to me: This is A LOT healthier
than conventional system, many people enjoy video games and with the Oculus it's going to be near perfect because the games that are and will be developed (I'm assuming) will also have walking involved, so a person will be exercising. There's also a person moving their head more than just their head movement being limited to a smaller area. People will be having so much fun and not knowing that they're also keeping their bodies active. For me, personally, the free walking is what I want to do but... I'm not sure how that'll work O.o All of those looked great though, I guess it's up to the gamers to decide in their future (when these things become more available) what they'll use for their movement.

Yeah I know what you mean - computing has been regarded as an unhealthy passtime ever since it first came about, and I feel like with the advent of VR and the increasing demand for virtual movements to be triggered by real-life material movement we'll find that people who sit inside all day playing video games will wind up quite a bit healthier than the average joe! Depending on the games they play, of course. 😉
I wish free-walking was a realistic option but the sad reality is that it's just not going to be feasible for everybody, which is what I think we really need sometime very soon - a system that does everything we need it to, that everyone can use.

"kwright62" wrote:
The VR walker combined with a 3D "terrain" that mimics the VR would be the ultimate amusement park
The Omni or similar input device will be for higher end systems
Kinect or similar sensor will allow standing in place and running in place to serve as inputs

Hmm, I don't think that Project VR Walker is for real, or at least, I don't think that it'll ever get off the ground. Call me a naysayer, but it just doesn't look feasible. I'm keeping an eye on it in case anything changes, however.
I do think that 'ODT's (I really wish Omni hadn't coined that term, since they're not treadmills at all, but whatever) are going to become pretty widely used - enough people have bought Omnis and Virtualizers for there to be enough support for additional units. Hopefully they can get the footprint and cost down, and maybe even solve the waist-strap problem!
And I hate to say it, but I disagree with you on the last point - running in place will always feel a little goofy. At best, it's a 'for now' solution while we wait for a more immersive locomotion controller. 😛

"mptp" wrote:
I really wish Omni hadn't coined that term, since they're not treadmills at all, but whatever

I thought that was an existing term?

"mptp" wrote:

There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.

Thanks for making the detailed write up on the current state of "walkers" as you call it.
The title of the post was really off-putting, but it was worth the read :lol:

I do agree that the current state has plenty of room for improvement, on many levels.
So there are many challenges that need to be worked on.

I think I should give a heads up that lowv posted an idea for a low cost device over at mtsb,
which as I suggested, he is going to make into an open-source hardware project.
I think it is a new approach to the problem, which has some potential,
and there should be plenty of incentive for those interested to take part in the project, so do chime in.
http://www.mtbs3d.com/phpBB/viewtopic.php?f=145&t=19932

As a matter of fact, I am also working on a separate affordable "locomotion" solution for people in normal room settings. It is in early development stage, and I am contemplating turning it into an open-source hardware project too. So I hope this space will be seeing some interesting news to come :!: 😄

edit: I just realized that the date of both of the OP posts are exactly the same date,
so we have witnessed a synchronicity event in the collective unconscious, or the Hundredth monkey effect :!: :idea:

further edit: just realized you already discussed lowvs idea on this forum!, so maybe you should update the post to reflect that?

You know that's a good point, I didn't think to include LocoVR in this list for some reason - especially now that the idea has been fleshed out a whole lot more since it was originally posted in Input Devices. 😄

I read through the post, but I didn't see anyone mention wireless locomotion.

We developed a high accuracy indoor tracking system that uses wireless RF tags. Each tag is about 1" x 0.5" and can transmit the absolute location.

Right now we are testing the Gear VR with the indoor tracking system and so far it seems to work really well.

Is anyone else working on something like this?

Michael

I would LOVE to see an RFID demo, if you guys have any. It was a system that I was going to tacking inside my VRgun for positional tracking. Will the system work well with Atmel Chipsets? Or would I have to go with something with more beef? right now I am tracking optically, but I have been concerned with effecting the Rift in a negative way with my IR lights, so when I was looking for solutions this came up, but I was worried about speed, and tracking areas. Could you point me in the right direction? We are currently trying anything that we can to find a good VR input!

~B

The indoor tracking system we are working on is meant to track the person's position as they walk around an indoor or outdoor space. This allows them to get up and walk around in the virtual world they create. We are going to make it open source so people can experiment and develop new ideas with it.

The tracking tag already has a ARM F4 processor onboard that handles all the math to calculate the absolute position. It transmits this information either by hardwired USB or wirelessly via Bluetooth for example to the Note 4 inside the gear VR.

Anyway, I think you are needing something different. You are wanting to track the position of the gun with respect to your Rift which is best handled optically.