Forum Discussion
Cross-fading Acceleration and retargeting gravity
I posted a wall of text here on Reddit:
http://www.reddit.com/r/oculus/comments/2c4bzv/crossfading_to_avoid_sickness_and_redirecting
Quoted here for convenience:
I had two "brain storming" thoughts recently and I'm curious if anyone has tried anything similar at all yet. As we all know, travel sickness occurs when your eyes and ears disagree on the acceleration (not velocity) occurring in the world, hence why VR comfort mode has been proposed. It is, in essence, the opposite of the ambition of filmmakers to use static images in quick succession to create motion: we must reverse that process and find a way of taking two scenes where the velocity and position have changed and convince the eyes that this happened with no acceleration!
One idea I had was to cross-fade two scenes: so for example a downhill skiing game could crossfade a scene at 0 mph to a 40 mph constant downhill velocity. The problem with doing that is you have to render two full scenes... in stereo... alpha blend... barrel distort, so you are effectively doubling the render cost (hey, can your scene render at 1000/74/2 ms?). So maybe this could only be done in very low poly areas. In the downhill skiing example, maybe the top of the hill could be purposely low poly. Another idea would be to combine the barrel distortion and crossfade tasks, but that's only going to save a millisecond or so.
Of course maybe it doesn't have to be a fade: maybe a dissolve, or more usefully maybe a vertical or horizontal wipe transition could be enough, meaning that you could use frustum culling to avoid rendering everything twice. On the transition line though you would end up rendering stuff twice though, and maybe a wipe wouldn't be as comfortable as a crossfade anyway. Would love to see a demo which compares though! I guess it could be a competition: put Brendan or someone else who suffers badly from simulator sickness and see how many 0 to 30mph transitions they can handle with each method! (For science!)
How long does the fade transition have to be in frames though? Well again this is probably where experimentation comes in, but you could guess that maybe it's the opposite of the illusion of motion. I hate the 24fps figure that is thrown around: I would say that a N64 gamer knows well that 18fps can be interpreted as motion! ;-) But we all agree that 5fps is something of a slideshow! So maybe 75/5 means a 15 frame transition is as a good starting point?
The second brain storming idea I had was to do with Retargetting Acceleration. So I'm guessing that the human ear is at the Wiimote level and not the Rift tracker tech level. It has fluid in it which can detect acceleration or gravity and it uses the rest of out bodies senses to fill in the gaps: e.g. if it feels 1G upwards pressure on your feet and a 1G downward sensation in your ears, it guesses you are standing up straight and not moving.
Now maybe it's possible in VR to redirect this gravity vector and pretend it is something else. So for example if you wore the Rift lying back at 45 degrees, maybe you could show a scene of the player orientated upright in the Rift. They then have 0.707G vertical component pulling down, but also an unexplained 0.707G pulling backwards. By showing a scene of them moving forwards (and downwards?) maybe you could give the ear an explanation of the 0.707G and so swap gravity for acceleration. Who knew the curvature of spacetime was relevant to VR huh? ;-)
There you go, braindump complete. Interested to hear feedback!
http://www.reddit.com/r/oculus/comments/2c4bzv/crossfading_to_avoid_sickness_and_redirecting
Quoted here for convenience:
I had two "brain storming" thoughts recently and I'm curious if anyone has tried anything similar at all yet. As we all know, travel sickness occurs when your eyes and ears disagree on the acceleration (not velocity) occurring in the world, hence why VR comfort mode has been proposed. It is, in essence, the opposite of the ambition of filmmakers to use static images in quick succession to create motion: we must reverse that process and find a way of taking two scenes where the velocity and position have changed and convince the eyes that this happened with no acceleration!
One idea I had was to cross-fade two scenes: so for example a downhill skiing game could crossfade a scene at 0 mph to a 40 mph constant downhill velocity. The problem with doing that is you have to render two full scenes... in stereo... alpha blend... barrel distort, so you are effectively doubling the render cost (hey, can your scene render at 1000/74/2 ms?). So maybe this could only be done in very low poly areas. In the downhill skiing example, maybe the top of the hill could be purposely low poly. Another idea would be to combine the barrel distortion and crossfade tasks, but that's only going to save a millisecond or so.
Of course maybe it doesn't have to be a fade: maybe a dissolve, or more usefully maybe a vertical or horizontal wipe transition could be enough, meaning that you could use frustum culling to avoid rendering everything twice. On the transition line though you would end up rendering stuff twice though, and maybe a wipe wouldn't be as comfortable as a crossfade anyway. Would love to see a demo which compares though! I guess it could be a competition: put Brendan or someone else who suffers badly from simulator sickness and see how many 0 to 30mph transitions they can handle with each method! (For science!)
How long does the fade transition have to be in frames though? Well again this is probably where experimentation comes in, but you could guess that maybe it's the opposite of the illusion of motion. I hate the 24fps figure that is thrown around: I would say that a N64 gamer knows well that 18fps can be interpreted as motion! ;-) But we all agree that 5fps is something of a slideshow! So maybe 75/5 means a 15 frame transition is as a good starting point?
The second brain storming idea I had was to do with Retargetting Acceleration. So I'm guessing that the human ear is at the Wiimote level and not the Rift tracker tech level. It has fluid in it which can detect acceleration or gravity and it uses the rest of out bodies senses to fill in the gaps: e.g. if it feels 1G upwards pressure on your feet and a 1G downward sensation in your ears, it guesses you are standing up straight and not moving.
Now maybe it's possible in VR to redirect this gravity vector and pretend it is something else. So for example if you wore the Rift lying back at 45 degrees, maybe you could show a scene of the player orientated upright in the Rift. They then have 0.707G vertical component pulling down, but also an unexplained 0.707G pulling backwards. By showing a scene of them moving forwards (and downwards?) maybe you could give the ear an explanation of the 0.707G and so swap gravity for acceleration. Who knew the curvature of spacetime was relevant to VR huh? ;-)
There you go, braindump complete. Interested to hear feedback!
6 Replies
- I like your thinking. I wonder if a cushion you sit on that can slide around like two plates on one another, would give enough movement to the inner ear to think things were ok since it's probably just the initial acceleration. This would also help with immersion and could even be used as a force feedback device.
"davehampson" wrote:
Now maybe it's possible in VR to redirect this gravity vector and pretend it is something else. So for example if you wore the Rift lying back at 45 degrees, maybe you could show a scene of the player orientated upright in the Rift. They then have 0.707G vertical component pulling down, but also an unexplained 0.707G pulling backwards. By showing a scene of them moving forwards (and downwards?) maybe you could give the ear an explanation of the 0.707G and so swap gravity for acceleration. Who knew the curvature of spacetime was relevant to VR huh? ;-)
There you go, braindump complete. Interested to hear feedback!
There is another idea related to this, I think it was mentioned before.
When accelerating forward you slightly rotate the camera downward. we tend to unconsciously compensate for this unexpected rotation by moving our head up. (We automatically look to center our view at the horizon). Since our sense of acceleration is mainly in the inner ear we should then "think" we are accelerating forward according to the discrepancy.
This has the advantage of not requiring any extra peripherals, but it could only work within subtle amounts (ie walking speed) This is obviously a slight deviation from the "best practices" of 1:1 correspondence between head and camera rotation, but I think depending on the input method it might be more comfortable to use.
We might be able to tolerate small discrepancies in acceleration and rotation better than no discrepancy in rotation and a large discrepancy in acceleration.
Even more our sense of direction is a relative one in the centre of our view (derived by vision, memory and the strain on the neck) while acceleration is an absolute one (directly from the inner ear)."ElectricMucus" wrote:
"davehampson" wrote:
Now maybe it's possible in VR to redirect this gravity vector and pretend it is something else. So for example if you wore the Rift lying back at 45 degrees, maybe you could show a scene of the player orientated upright in the Rift. They then have 0.707G vertical component pulling down, but also an unexplained 0.707G pulling backwards. By showing a scene of them moving forwards (and downwards?) maybe you could give the ear an explanation of the 0.707G and so swap gravity for acceleration. Who knew the curvature of spacetime was relevant to VR huh? ;-)
There you go, braindump complete. Interested to hear feedback!
There is another idea related to this, I think it was mentioned before.
When accelerating forward you slightly rotate the camera downward. we tend to unconsciously compensate for this unexpected rotation by moving our head up. (We automatically look to center our view at the horizon). Since our sense of acceleration is mainly in the inner ear we should then "think" we are accelerating forward according to the discrepancy.
I think the problem there is that even though we would tilt our head up unconsciously, we probably detect acceleration and orientation at that same unconscious level as well, so it may not work.
Making a gimbal that smoothly rotated the players entire body would probably work. That's pretty expensive though ;)"davehampson" wrote:
I think the problem there is that even though we would tilt our head up unconsciously, we probably detect acceleration and orientation at that same unconscious level as well, so it may not work.
The only way to know is to try it out. I'll probably give it a go when I get my DK2.
It all depends on how much the sense of direction can be effectively fooled without discomfort. Acceleration and direction aren't instantaneous perceptions like vision or hearing but are more like perception over time. Perception in general is more subject to the expectations then we realise."davehampson" wrote:
I'd say that's in the will work 'for sure' category since that technique has been used in military training flight simulators for decades. :)
Making a gimbal that smoothly rotated the players entire body would probably work. That's pretty expensive though ;)"ElectricMucus" wrote:
"davehampson" wrote:
I think the problem there is that even though we would tilt our head up unconsciously, we probably detect acceleration and orientation at that same unconscious level as well, so it may not work.
The only way to know is to try it out. I'll probably give it a go when I get my DK2.
The thinking is pretty valid - the eyes coordinate with the inner ear to determine normal gravity orientation. When the eyes are brought back "on target" they do so at the cost of the relative vestibular reference, which may end up being interpreted as a directional shift in said reference. I was actually wondering along these same lines, if you could insert a small rate of error for a combined effect of vestibular redirection, but also some static positional fatigue - not usually desirable, but in the case of simulating cockpit G-forces may be able to add to the realism.
Probably not the best idea, though - people get air sick when getting used to those kinds of forces and effects in real life, without any help from a VR intermediary. It might work for veterans, but would almost certainly make sim-sickness worse for the general populace."davehampson" wrote:
The second brain storming idea I had was to do with Retargetting Acceleration. So I'm guessing that the human ear is at the Wiimote level and not the Rift tracker tech level. It has fluid in it which can detect acceleration or gravity and it uses the rest of out bodies senses to fill in the gaps: e.g. if it feels 1G upwards pressure on your feet and a 1G downward sensation in your ears, it guesses you are standing up straight and not moving.
Now maybe it's possible in VR to redirect this gravity vector and pretend it is something else. So for example if you wore the Rift lying back at 45 degrees, maybe you could show a scene of the player orientated upright in the Rift. They then have 0.707G vertical component pulling down, but also an unexplained 0.707G pulling backwards. By showing a scene of them moving forwards (and downwards?) maybe you could give the ear an explanation of the 0.707G and so swap gravity for acceleration. Who knew the curvature of spacetime was relevant to VR huh? ;-)
I also brainstormed that same idea before on this forum, as one of my ideas for preventing motion sickness.
But I think the human ear is at the Wii Motion Plus level, rather than just the normal Wiimote. ie. I think humans have gyros that measure angular velocity, not just accelerometers. So it might not quite work. My idea was to tilt the view sideways while virtually accelerating sideways, so that gravity was being used to simulate the acceleration in your inner ear. But if that upsets your gyros, then it might make you motion sick in a different way.
