Walking in VR [Long, Long, Long Post]
I keep finding myself posting really long posts in random threads that start talking about locomotion in VR, so I'm going to make a big post here instead that I can just point to.
Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.
The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:
Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.
Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?
Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:
There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:
So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.
Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about
The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:
I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.
What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.
Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.
The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:
- The Virtuix Omni
The first of the harness-style omnidirectional treadmills. A curved, low-friction surface, special shoes, and a weight-removing harness allow the user to convert the natural walking and running motions they use everyday into virtual movement commands. - The Cyberith Virtualizer
Following the Omni's example, the Virtualizer works similarly to the Omni, but allows the user to crouch and jump while remaining in the harness, allows the user to play in socks rather than special shoes, and has a flat surface rather than the Omni's curved one. - The Walkmouse
A large-circular panel inset with many small rollers, this treadmill simulates a moving surface over which the user can walk. - The StinkyBoard
Effectively a four-button gamepad for a single foot, this device allows the user to emulate WASD keyboard events using the feet, and so achieve simple VR locomotion - The Wii BalanceBoard
Not actually a PC peripheral, with some hacking can be used as a lean-controller, allowing the user to move in VR by leaning their body forward, backwards, left and right - Stompz
Implements IMUs on each feet to allow the user to make 'stomping' gestures to move forward, left and right in game - Infinideck
A more traditional-style treadmill that can move in any 2D direction, that the user walks on as they would walk on a normal surface. Unfortunately, must be manually controller by a second person at the time of this writing - WizDish
A low-friction surface that removes the vertical foot-motion component of walking, leaving the user to make skii-ing like gestures to control motion. At the time of this writing, lacks community support and/or funding - Free-Walking
Perhaps the most immersive of them all - the user walks around a physically-large space (like a football field or warehouse), with real walked distance corresponding 1:1 to virtual walked distance - LocoVR
A crowd-sourced project involving what is described as its conceiver as "basically a cross between a small bar stool and a kind of oversized joystick", able to rotate freely and respond to lean from the user to provide locomotion input - Pointman
In use by the US Navy for infantry simulation, the Pointman system is actually a combination of several input devices. One of them is a device comprised of two footpedals able to move forward and backwards, as well as to be pressed down like a car acceleration pedal. At the time of this writing, a handheld controller is required for turning.
Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.
- Price
This one is a bit of a no-brainer. The cheaper something is, the more people will buy it. I would say that once the price of a device goes above about $150, it becomes a serious barrier to widespread adoption. Not that the ideal VR walker should necessarily be cheaper than this, but it's a good ballpark figure to go for. - Size
Most people live in cities, meaning most people have very limited living space. That is to say, people tend to be less than impressed if asked to reserve an area the size of a double-bed for a VR input device. So the physical size of the device will be a further barrier to widespread adoption. Ideally, the system will fit onto the user, so it effectively takes up no space in and of itself. - Flexibility
Keep in mind that locomotion in games isn't always human-style walking/running. For example, think of Minecraft with a Virtuix Omni. The Omni would work great while you're running around mining, building and fighting zombies. But if you want to play in creative-mode, suddenly you have this big mismatch between flying around in-game, but making walking and running motions in reality to drive this motion. For a device to become widely-used, it needs to be suited to a wide variety of locomotion applications, not just walking and running
Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?
- Movement similarity
We all know what walking feels like. We know what the movements are, and how it feels when we perform them. The closer a VR Walker can get to emulating this motion, the more immersive it will be. In this way, devices like the Omni or the Virtualizer are a step-up from solutions like the StinkyBoard or Stompz. The obvious ideal is devices like the WalkMouse or InfiniDeck, the question of whether these solutions are currently realistic remains to be seen. - Constraints on movement
An important thing to consider is the degree to which a VR Walker inhibits our ability to perform actions unrelated to locomotion. For example, the Omni doesn't allow users to bend down, because they're stuck in a harness that has a fixed height off the ground. Both the Omni and the Virtualizer don't allow the user to lower their hands below their waist, since the harness gets in the way. These kinds of movement constraints are non-issues in some games, especially FPS, where the user always has their hands above their waist holding a gun, but significantly annoying in games involving peaceful exploration, where the user is likely to want their hands to rest in their natural position - down by their sides. - Strenuousness (Spellcheck agrees with me that this is a real word)
When we're talking about using natural motion to run around in games, obviously an issue is going to be people getting tired! I personally can't run for much more than 5 minutes before I'm completely wasted. I'm definitely not up to running around for a half-hour gaming session. This isn't only about physical capability - if people are getting tired and cramped while playing, this is an immersion-breaking experience and something to be avoided.
Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:
- Move the ground under the user
This is the solution that WalkMouse and InfiniDeck are pursuing. This is also the solution that traditional exercise treadmills implement. All you have to do is have a device that the user stands on, and when they move, detect the speed of movement, and move the surface they're standing on in the opposite direction at that speed, negating any physical movement the user might undergo, saving them from crashing into a wall and breaking their precious precious DK2. - Remove the frictional force
This is the solution that the Omni and the Virtualizer are using, and many hypothetical input devices involving suspending the user from harnesses on the ceiling etc. are also operating under this principle.
This solution typically involves both reducing the normal force (that is, taking the user's weight off the feet so that the feet are effectively suspended), and reducing the coefficient of friction between the feet and the ground (eg. socks on a polished wood floor) so that the feet can easily slide over the surface.
There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:
- Moving the ground under the user requires many moving parts, increasing cost and reducing reliability
- Ensuring that the dynamic surface behaves exactly as the user would expect a static surface to behave is 100% essential to avoid loss of balance leading to injury, and requires zero latency, which is an engineering challenge not likely to be solved any time soon
So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.
Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about
- IMUs (Inertial Measurement Units)
Specifically, I'm talking about IMUs containing an accelerometer, a gyroscope and a magnetometer. Through sensor-fusion these three devices allow us to get drift-free rotational measurements at about 1kHz. In addition, with the advent of MEMS technology, they're pretty cheap, and digital MEMS IMUs do much of the processing on the chip, so they're easy to use. - Optical tracking
Optical tracking is awesome because it gives us 1:1 positional data in realtime, and this data will never drift. The main Achilles' Heel of optical tracking is occlusion - any objects between the camera and the tracked objects will interfere with tracking. - Pressure sensors
These devices measure pressure applied downwards on them. Four of these are all the sensors that the Wii BalanceBoard has behind it. They're quite versatile, and can be small enough to be used in wearable devices. - Capacitative sensors
These are effectively touch-sensors, that when placed in an array, allow the position and shape of an object in contact with the sensor surface to be resolved. These are the sensors behind the Omni.
The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:
- What device removes the weight of the user off the ground so that they can use their normal walking/running motions, but doesn't impose any other movement restrictions? Does this device take up little room, and is it cheaper than $200? Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?
I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.
What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.