Walking in VR [Long, Long, Long Post]

[mptp]

I keep finding myself posting really long posts in random threads that start talking about locomotion in VR, so I'm going to make a big post here instead that I can just point to.

Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.

The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:

• The Virtuix Omni
  The first of the harness-style omnidirectional treadmills. A curved, low-friction surface, special shoes, and a weight-removing harness allow the user to convert the natural walking and running motions they use everyday into virtual movement commands.


• The Cyberith Virtualizer
  Following the Omni's example, the Virtualizer works similarly to the Omni, but allows the user to crouch and jump while remaining in the harness, allows the user to play in socks rather than special shoes, and has a flat surface rather than the Omni's curved one.


• The Walkmouse
  A large-circular panel inset with many small rollers, this treadmill simulates a moving surface over which the user can walk.


• The StinkyBoard
  Effectively a four-button gamepad for a single foot, this device allows the user to emulate WASD keyboard events using the feet, and so achieve simple VR locomotion


• The Wii BalanceBoard
  Not actually a PC peripheral, with some hacking can be used as a lean-controller, allowing the user to move in VR by leaning their body forward, backwards, left and right


• Stompz
  Implements IMUs on each feet to allow the user to make 'stomping' gestures to move forward, left and right in game


• Infinideck
  A more traditional-style treadmill that can move in any 2D direction, that the user walks on as they would walk on a normal surface. Unfortunately, must be manually controller by a second person at the time of this writing


• WizDish
  A low-friction surface that removes the vertical foot-motion component of walking, leaving the user to make skii-ing like gestures to control motion. At the time of this writing, lacks community support and/or funding


• Free-Walking
  Perhaps the most immersive of them all - the user walks around a physically-large space (like a football field or warehouse), with real walked distance corresponding 1:1 to virtual walked distance


• LocoVR
  A crowd-sourced project involving what is described as its conceiver as "basically a cross between a small bar stool and a kind of oversized joystick", able to rotate freely and respond to lean from the user to provide locomotion input


• Pointman
  In use by the US Navy for infantry simulation, the Pointman system is actually a combination of several input devices. One of them is a device comprised of two footpedals able to move forward and backwards, as well as to be pressed down like a car acceleration pedal. At the time of this writing, a handheld controller is required for turning.

Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.

• Price
  This one is a bit of a no-brainer. The cheaper something is, the more people will buy it. I would say that once the price of a device goes above about $150, it becomes a serious barrier to widespread adoption. Not that the ideal VR walker should necessarily be cheaper than this, but it's a good ballpark figure to go for.


• Size
  Most people live in cities, meaning most people have very limited living space. That is to say, people tend to be less than impressed if asked to reserve an area the size of a double-bed for a VR input device. So the physical size of the device will be a further barrier to widespread adoption. Ideally, the system will fit onto the user, so it effectively takes up no space in and of itself.


• Flexibility
  Keep in mind that locomotion in games isn't always human-style walking/running. For example, think of Minecraft with a Virtuix Omni. The Omni would work great while you're running around mining, building and fighting zombies. But if you want to play in creative-mode, suddenly you have this big mismatch between flying around in-game, but making walking and running motions in reality to drive this motion. For a device to become widely-used, it needs to be suited to a wide variety of locomotion applications, not just walking and running

Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?

• Movement similarity
  We all know what walking feels like. We know what the movements are, and how it feels when we perform them. The closer a VR Walker can get to emulating this motion, the more immersive it will be. In this way, devices like the Omni or the Virtualizer are a step-up from solutions like the StinkyBoard or Stompz. The obvious ideal is devices like the WalkMouse or InfiniDeck, the question of whether these solutions are currently realistic remains to be seen.


• Constraints on movement
  An important thing to consider is the degree to which a VR Walker inhibits our ability to perform actions unrelated to locomotion. For example, the Omni doesn't allow users to bend down, because they're stuck in a harness that has a fixed height off the ground. Both the Omni and the Virtualizer don't allow the user to lower their hands below their waist, since the harness gets in the way. These kinds of movement constraints are non-issues in some games, especially FPS, where the user always has their hands above their waist holding a gun, but significantly annoying in games involving peaceful exploration, where the user is likely to want their hands to rest in their natural position - down by their sides.


• Strenuousness (Spellcheck agrees with me that this is a real word)
  When we're talking about using natural motion to run around in games, obviously an issue is going to be people getting tired! I personally can't run for much more than 5 minutes before I'm completely wasted. I'm definitely not up to running around for a half-hour gaming session. This isn't only about physical capability - if people are getting tired and cramped while playing, this is an immersion-breaking experience and something to be avoided.

Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:

• Move the ground under the user
  This is the solution that WalkMouse and InfiniDeck are pursuing. This is also the solution that traditional exercise treadmills implement. All you have to do is have a device that the user stands on, and when they move, detect the speed of movement, and move the surface they're standing on in the opposite direction at that speed, negating any physical movement the user might undergo, saving them from crashing into a wall and breaking their precious precious DK2.


• Remove the frictional force
  This is the solution that the Omni and the Virtualizer are using, and many hypothetical input devices involving suspending the user from harnesses on the ceiling etc. are also operating under this principle.
  This solution typically involves both reducing the normal force (that is, taking the user's weight off the feet so that the feet are effectively suspended), and reducing the coefficient of friction between the feet and the ground (eg. socks on a polished wood floor) so that the feet can easily slide over the surface.

There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:

• Moving the ground under the user requires many moving parts, increasing cost and reducing reliability


• Ensuring that the dynamic surface behaves exactly as the user would expect a static surface to behave is 100% essential to avoid loss of balance leading to injury, and requires zero latency, which is an engineering challenge not likely to be solved any time soon

So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.

Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about

• IMUs (Inertial Measurement Units)
  Specifically, I'm talking about IMUs containing an accelerometer, a gyroscope and a magnetometer. Through sensor-fusion these three devices allow us to get drift-free rotational measurements at about 1kHz. In addition, with the advent of MEMS technology, they're pretty cheap, and digital MEMS IMUs do much of the processing on the chip, so they're easy to use.


• Optical tracking
  Optical tracking is awesome because it gives us 1:1 positional data in realtime, and this data will never drift. The main Achilles' Heel of optical tracking is occlusion - any objects between the camera and the tracked objects will interfere with tracking.


• Pressure sensors
  These devices measure pressure applied downwards on them. Four of these are all the sensors that the Wii BalanceBoard has behind it. They're quite versatile, and can be small enough to be used in wearable devices.


• Capacitative sensors
  These are effectively touch-sensors, that when placed in an array, allow the position and shape of an object in contact with the sensor surface to be resolved. These are the sensors behind the Omni.

The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:

What device removes the weight of the user off the ground so that they can use their normal walking/running motions, but doesn't impose any other movement restrictions? Does this device take up little room, and is it cheaper than $200? Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?

I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.

What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.

[Leahy]

"Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?"


That was a good post. You basically addressed most or near all of the issues we will be facing in the near future as a Walker gains traction in the consumer market. What I would like to see or I may even create is a 45° walker/harness/chair which I'd like to be able to adjust from 80°-30°. You add on 2D roller shoes which serve basically to adjust resistance and a platform which tilts with the incline (and possibly in-game tilt) and you have a Walker which can be used for most people for an indefinite period of time.

This of course will never meet your $150-$200 goal but I can see it coming close with widespread adoption. Which brings me to the crux of your post which I did not feel expressed strongly enough. Developers need to program for such a mechanism. Yet why bother when there is so little market? I felt really let down when seeing that the omni has WSAD translation instead of real foot placement which I understand is completely beyond the current realms of possibility in today's games.

Feel free to steal my idea and run with it. In the spirit of Oculus it takes a fair bit more than an idea to gain traction in the market. Maybe you'll be able to make it work where I can't.

[mystify]

"Leahy" wrote:
Yet why bother when there is so little market? I felt really let down when seeing that the omni has WSAD translation instead of real foot placement which I understand is completely beyond the current realms of possibility in today's games.

I thought it had both, or has that changed? WASD capabilities are pretty essential for generic use, but I thought it also allowed access to more precise data for things designed to work with it.

[Leahy]

I'm pretty sure it does have both. The thing is what games are made to accept more precise input when such a infinitesimal percent of users might require more precise input

[mystify]

"Leahy" wrote:
I'm pretty sure it does have both. The thing is what games are made to accept more precise input when such a infinitesimal percent of users might require more precise input

what games are made to support VR when such an infitesimal percentage of users may have the equipment?
More specific to this situation, if the omni can get enough people to possess one, then it creates an incentive for people to code directly for it. In order to do that, they have to get as much compatability as they can without people coding for them, and hence WASD support is vitally important. This is a distinct advantage over a motion controller like the hydra, which really needs to be coded for to be worth anything, while being able to walk around in VR at all is a big step.

[Leahy]

Yeah it's a chicken-egg sort of problem. I'm not disagreeing at all. The hydra actually can be fairly useful once you finally hop over the big learning curve of motioncreator, still it took me months to finally sit down and dedicate a few afternoons to do so. I really don't wish to talk out of my a** since I haven't tried better but I cannot help but think that floating would be less immersive than individual steps. I know something is better than nothing. I'm just plagued by looking a bit much into the future as is common in these forums

[OculusRiftRocks]

Could you not combine the harness of the omni with roller skates on a standard floor surface.

[mystify]

"OculusRiftRocks" wrote:
Could you not combine the harness of the omni with roller skates on a standard floor surface.

Then you wouldn't have the motion tracking of your feet. Plus, the omni has a curve that is supposed to make the walk feel more natural, and the roller blades would act weird when you turn, or if your foot twists

[OculusRiftRocks]

people say walking on the omni feels like using rollerblades.

And yeah, I was referring to excluding the tracking pods, just the omni itself.

[konstantin_loze]

Congratulations for the topic, well thought and written. I have been asking myself the same questions and I think I came up with the cheapest possible solution. It is by no means great and for VR it will require a lot of tweaking, but the idea is simple. I actually did it in the span of a late Sunday afternoon. I put some velcro on the ceiling (wife does not allow drilling 🙂 ) then attached an old android phone (Xperia mini pro) with installed DroidCam app that acts as a webcam over the home wifi. I installed the PC app and checked that it functions with Freetrack. I then simply put a single 3 EUR bike LED on a cap and calibrated Freetrack with this one point model. Why one point model? Because the FOV of the ceiling-mounted android phone is not so large and a 3 point model exits the view much faster than a simple one LED model. Then I wrote a very simple GlovePIE script that translates TrackIR (as sent by Freetrack) yaw and pitch into WASD and I added a deadzone and a variable counter corresponding to the edge offset (the closer to the edge of the Freetrack camera, the shorter the counter) that triggers WASD upon resetting itself. Freetrack pitch and yaw based on the one point model are not linear, but that is even better, since that provides a nice transition from the deadzone. The Freetrack output curves can be tweaked additionally to your linking.
I copy below the link to the youtube video, but please note this is not a listed video, since I intend to make a better one. This is only a proof of concept.

https://www.youtube.com/watch?v=1fSFNb5QWqY

[OculusRiftRocks]

pretty cool idea but sadly can't turn in that case.