Walking in VR [Long, Long, Long Post]

[mptp]

I keep finding myself posting really long posts in random threads that start talking about locomotion in VR, so I'm going to make a big post here instead that I can just point to.

Introduction
There are lots of problems to be solved in VR. Perhaps the single most critical problem right now is the question of Input. For the purposes of this thread, I'm going to take it as a given that we need the ability to reserve our hands for doing hand-related things, and not for moving around. That is to say, we're going to need to use our feet or legs to operate the device that will allow us to move around in VR. I've been calling these locomotion devices 'Walkers'.
I should point out that I'm writing this thing for anyone to read. I realise that most of the people here will know what optical-tracking occlusion is, or well-understand the downsides to the Virtuix Omni, but I'm putting it all here just to be thorough.

The Current State of VR Walkers
Most people on this forum will be well-versed on the current state of VR input, but just in case anyone missed a device or two, I'm going to put here a list of all VR Walkers that I'm currently aware of. If I miss any, let me know and I'll add it asap.
In no particular order:

• The Virtuix Omni
  The first of the harness-style omnidirectional treadmills. A curved, low-friction surface, special shoes, and a weight-removing harness allow the user to convert the natural walking and running motions they use everyday into virtual movement commands.


• The Cyberith Virtualizer
  Following the Omni's example, the Virtualizer works similarly to the Omni, but allows the user to crouch and jump while remaining in the harness, allows the user to play in socks rather than special shoes, and has a flat surface rather than the Omni's curved one.


• The Walkmouse
  A large-circular panel inset with many small rollers, this treadmill simulates a moving surface over which the user can walk.


• The StinkyBoard
  Effectively a four-button gamepad for a single foot, this device allows the user to emulate WASD keyboard events using the feet, and so achieve simple VR locomotion


• The Wii BalanceBoard
  Not actually a PC peripheral, with some hacking can be used as a lean-controller, allowing the user to move in VR by leaning their body forward, backwards, left and right


• Stompz
  Implements IMUs on each feet to allow the user to make 'stomping' gestures to move forward, left and right in game


• Infinideck
  A more traditional-style treadmill that can move in any 2D direction, that the user walks on as they would walk on a normal surface. Unfortunately, must be manually controller by a second person at the time of this writing


• WizDish
  A low-friction surface that removes the vertical foot-motion component of walking, leaving the user to make skii-ing like gestures to control motion. At the time of this writing, lacks community support and/or funding


• Free-Walking
  Perhaps the most immersive of them all - the user walks around a physically-large space (like a football field or warehouse), with real walked distance corresponding 1:1 to virtual walked distance


• LocoVR
  A crowd-sourced project involving what is described as its conceiver as "basically a cross between a small bar stool and a kind of oversized joystick", able to rotate freely and respond to lean from the user to provide locomotion input


• Pointman
  In use by the US Navy for infantry simulation, the Pointman system is actually a combination of several input devices. One of them is a device comprised of two footpedals able to move forward and backwards, as well as to be pressed down like a car acceleration pedal. At the time of this writing, a handheld controller is required for turning.

Adoption Factors
A critical thing with input devices is the degree of adoption. We obviously want a device that will be used by as many people as possible - this will give developers confidence that they aren't wasting their time by building games that support this hypothetical device, and with more content for the device, adoption will increase further, until the device is largely ubiquitous.

• Price
  This one is a bit of a no-brainer. The cheaper something is, the more people will buy it. I would say that once the price of a device goes above about $150, it becomes a serious barrier to widespread adoption. Not that the ideal VR walker should necessarily be cheaper than this, but it's a good ballpark figure to go for.


• Size
  Most people live in cities, meaning most people have very limited living space. That is to say, people tend to be less than impressed if asked to reserve an area the size of a double-bed for a VR input device. So the physical size of the device will be a further barrier to widespread adoption. Ideally, the system will fit onto the user, so it effectively takes up no space in and of itself.


• Flexibility
  Keep in mind that locomotion in games isn't always human-style walking/running. For example, think of Minecraft with a Virtuix Omni. The Omni would work great while you're running around mining, building and fighting zombies. But if you want to play in creative-mode, suddenly you have this big mismatch between flying around in-game, but making walking and running motions in reality to drive this motion. For a device to become widely-used, it needs to be suited to a wide variety of locomotion applications, not just walking and running

Immersion Factors
So, casting aside questions of what will get the device used, and turning our attention instead to what will make the device great, we need to think about what factors actually influence how suited the device is for VR. That is to say, how immersive is the device to use?

• Movement similarity
  We all know what walking feels like. We know what the movements are, and how it feels when we perform them. The closer a VR Walker can get to emulating this motion, the more immersive it will be. In this way, devices like the Omni or the Virtualizer are a step-up from solutions like the StinkyBoard or Stompz. The obvious ideal is devices like the WalkMouse or InfiniDeck, the question of whether these solutions are currently realistic remains to be seen.


• Constraints on movement
  An important thing to consider is the degree to which a VR Walker inhibits our ability to perform actions unrelated to locomotion. For example, the Omni doesn't allow users to bend down, because they're stuck in a harness that has a fixed height off the ground. Both the Omni and the Virtualizer don't allow the user to lower their hands below their waist, since the harness gets in the way. These kinds of movement constraints are non-issues in some games, especially FPS, where the user always has their hands above their waist holding a gun, but significantly annoying in games involving peaceful exploration, where the user is likely to want their hands to rest in their natural position - down by their sides.


• Strenuousness (Spellcheck agrees with me that this is a real word)
  When we're talking about using natural motion to run around in games, obviously an issue is going to be people getting tired! I personally can't run for much more than 5 minutes before I'm completely wasted. I'm definitely not up to running around for a half-hour gaming session. This isn't only about physical capability - if people are getting tired and cramped while playing, this is an immersion-breaking experience and something to be avoided.

Friction
So that's the groundwork laid out. But the real question is 'what are the constraints under which the creator of a VR Walker must operate?'. The central constraint (if we want to use natural walking/running motion) is friction. More specifically, it's the normal component of friction.
When we walk, we are effectively applying pressure down onto the ground with our weight, and then using friction to use that weight to move us forward. If we want the users to be making the same motions in VR that they make in material reality (MR?), then we need some way of negating this forward motion. There are basically two ways of doing this:

• Move the ground under the user
  This is the solution that WalkMouse and InfiniDeck are pursuing. This is also the solution that traditional exercise treadmills implement. All you have to do is have a device that the user stands on, and when they move, detect the speed of movement, and move the surface they're standing on in the opposite direction at that speed, negating any physical movement the user might undergo, saving them from crashing into a wall and breaking their precious precious DK2.


• Remove the frictional force
  This is the solution that the Omni and the Virtualizer are using, and many hypothetical input devices involving suspending the user from harnesses on the ceiling etc. are also operating under this principle.
  This solution typically involves both reducing the normal force (that is, taking the user's weight off the feet so that the feet are effectively suspended), and reducing the coefficient of friction between the feet and the ground (eg. socks on a polished wood floor) so that the feet can easily slide over the surface.

There are advantages and disadvantages to both of these methods. For the following reasons, I'm going to be considering only the second option:

• Moving the ground under the user requires many moving parts, increasing cost and reducing reliability


• Ensuring that the dynamic surface behaves exactly as the user would expect a static surface to behave is 100% essential to avoid loss of balance leading to injury, and requires zero latency, which is an engineering challenge not likely to be solved any time soon

So the question now becomes 'how do we take the weight off the feet to allow natural input, without implementing a large mechanism to support a harness that limits the user's natural movement'?
Quite the brain teaser.

Sensors
The last thing I'll talk about is sensors, and which sensors are most applicable to the creation of a VR walker, and some considerations for using each one. I'm not really a Maker, so I'm just going to give an outline of a few that I've thought about

• IMUs (Inertial Measurement Units)
  Specifically, I'm talking about IMUs containing an accelerometer, a gyroscope and a magnetometer. Through sensor-fusion these three devices allow us to get drift-free rotational measurements at about 1kHz. In addition, with the advent of MEMS technology, they're pretty cheap, and digital MEMS IMUs do much of the processing on the chip, so they're easy to use.


• Optical tracking
  Optical tracking is awesome because it gives us 1:1 positional data in realtime, and this data will never drift. The main Achilles' Heel of optical tracking is occlusion - any objects between the camera and the tracked objects will interfere with tracking.


• Pressure sensors
  These devices measure pressure applied downwards on them. Four of these are all the sensors that the Wii BalanceBoard has behind it. They're quite versatile, and can be small enough to be used in wearable devices.


• Capacitative sensors
  These are effectively touch-sensors, that when placed in an array, allow the position and shape of an object in contact with the sensor surface to be resolved. These are the sensors behind the Omni.

The Challenge
So I hope I've summed up the state of VR Locomotion Input decently. I think what it boils down to is the following challenge:

What device removes the weight of the user off the ground so that they can use their normal walking/running motions, but doesn't impose any other movement restrictions? Does this device take up little room, and is it cheaper than $200? Can it be used to move around in experiences that don't necessarily involve walking/running? Can it be used for multiple hours at a time without the user getting tired or sore?

I don't think it's going to be an easy nut to crack, and I don't think any of the current solutions that are either on the market, or will be coming to market in the foreseeable future are adequate.

What do you guys think - I reckon as a community we can come up with some pretty sweet ideas that will hopefully be picked up by someone with the know-how and capital to make this ideal VR-walker a reality. I think it's the one thing that stands between us and totally immersive first-person walking VR experiences.
From what people are saying, Elite:Dangerous is mind-blowing because we finally have an experience where we can match input devices to virtual reality perfectly. Once we can do that with common non-cockpit experiences, VR will have taken a huge step forward.

[konstantin_loze]

Well, not with that script, but it is still, very much possible. For example, if your Rift is facing 90 degrees (as per the magnetometer corrected yaw reported by the Rift's IMU) to the right and you step forward, the script will have to be set up to translate the side offset of FreeTrack as if it was a vertical offset and press W instead of D (it will require some more coding for that correction though).
So, there is no reason why the absolute position used here cannot be translated in the reference framework of the Rift, if there is no yaw drift of the HMD's IMU solution. My problem is that I only have a very basic DIY HMD that does not have a magnetometer for correcting yaw drift (I am actually embarrassed to show it on youtube, it's very generic and not a pretty thing at all :)).
Still, I have experimented with a 3-point model facing up instead on a one-point as shown on the video, which tracks your head's yaw (FreeTack sees it as roll, of course) and eliminates the drift, but it has limitations on the FreeTrack software, which goes crazy if you try to turn more than 40 degrees. See this for example, but imagine the 3-point model is on your head facing up the ceiling mounted camera https://www.youtube.com/watch?v=MyeNllhpjfA
It should still be possible to use it for a simple WASD determination in combination with non-MARG setup (like mine).
Still, my immediate priority now is to combine this with sensing your footsteps, similar to Stompz, but tying a wiimote to your hip. This enhancement should really make a difference to the predictability and control to your movements and would require even less FOV of the ceiling-mounted camera.
I will keep everyone here posted on the progress, but things look bright so far 🙂

[konstantin_loze]

Hi, evrypne, I promised I will post progress on the WASD implementation with a ceiling-mounted mobile phone cam and walking trigger with a wiimote tied to your hip. Here it is, I will soon update the description to add a link to the script: https://www.youtube.com/watch?v=2qi7BR1owiQ . The script requires only a regular wiimote, no need for a motion plus 🙂 It can work with any android phone and FreePIE instead (even the cheapest ones have accelerometer) with some tweaking. I managed to implement also crouch and jump, for more exercise 🙂
I have left now only the part that swaps the direction based on you HMD's yaw. When I make a decent video showing the final result with proper caibration, I will post a separate topic. What I like about the setup is that it is really, really affordable and there is no soldering, etc. required.

[OculusRiftRocks]

I think with what you have said I may have found the best way to track your feet, possibly better than stem as you won't have to wear anything.

if we combine foot sensors on the edge of a slippery bowl along with the rifts builtin tracking to know which way you are facing, then you should be able to walk in all directions!

Would that not be the best solution for tracking your feet, even better than stem!

edit: foot sensors can be a dance pad underneath the carpet.

[konstantin_loze]

You mean one single ring-like dance pad with the same diameter as the bowl? I have never seen such... Otherwise it's definitely doable. I don't know how big and deep your bowl is and how you walk in it, so this is really hypothetical until you try.

[OculusRiftRocks]

my bowl is 80cm, the dance mat is about 80cm, when I walk forward I am on the edge of the bowl where the dance mat would pickup the pressure of my feet and turn it into buttons.

when I stop walking my feet naturally return to the center of the bowl where there is no feet tracking, so it should work perfectly.

as far as strafing goes, I don't think it is something that would work well anyway, even if it could be tracked correctly, the motion of strafing doen'st work very well with an ODT, the most important thing ni games would be to have independant hip tracking, gun face wouldn't bother me at all, but with hip tracking you could look in a direction where you not walking, with that you wont ever need to strafe.

[konstantin_loze]

Dance mat rectangular + soup bowl circular = uneven sensor area around the edge as you turn.

[nomotonous]

I doubt that there would be a day where there is widespread adoption on a 150 dollars device that only have support for some games and is used for 1 hour max. I can see the standard device being something like the Stinkyboard or a USB foot pedal that costs ~$30.

[OculusRiftRocks]

"konstantin_lozev" wrote:
Dance mat rectangular + soup bowl circular = uneven sensor area around the edge as you turn.

it worked out really well, the outer edges of the mat fit in with the bowl, I just used xpadder and set all the buttons to W.

[konstantin_loze]

I'd like to see a youtube video, please.

[OculusRiftRocks]

just watch any virtuix omni video made in 2014 , the prototype then used pressure pads for walking motion.

this it the dance mat I am using.



do you think it would be possible to change what freepie does with the right and left pressure pads based on what you were saying with determining the direction I am facing with the rifts built in IMU?

the dance mat is seen as a simple gamepad device.