Hacking Our Senses

[wildermuthn]

The Oculus Rift has created the virtual sense of sight, not perfectly, perhaps not even 'well' (relative to the human retina), but it has done so well enough. People step into the game because our sense of sight has been successfully virtualized.

But there are other senses to deal with. Hearing has already been taken care of — headphones and stereo sound work just fine. The Rift + Headphones makes for a truly immersive experience. But there's other senses that we know are a big problem: touch and movement. Lack of movement causes naseau. Lack of touch keeps us from truly entering into our virtual realities.

But the brain is smart. We can hack the mind's senses.

"Touch to touch sensory substitution is where information from touch receptors of one region can be used to perceive touch in another. For example, in one experiment by Bach-y-Rita, the touch perception was restored in a patient who lost peripheral sensation from leprosy. For example, this leprosy patient was equipped with a glove containing artificial contact sensors coupled to skin sensory receptors on the forehead (which was stimulated). After training and acclimation, the patient was able to experience data from the glove as if it was originating in the fingertips while ignoring the sensations in the forehead. After two days of training one of the leprosy subjects reported "the wonderful sensation of touching his wife, which he had been unable to experience for 20 years."

http://en.wikipedia.org/wiki/Sensory_substitution#Tactile.E2.80.93tactile_substitution_to_restore_pe...

The sensation of touch in VR may not require full-body suits that actually recreate the sensation of touch. A crude, but effective, implementation might only take an armband that recreates full-body touch on a small section of skin. The same could be done for a sense of movement. Yes, it would only be an approximation. But it might be just good enough to fool our minds.

But why stop with touch and movement? Why not give human beings entirely new senses?

"See with your tongue. Navigate with your skin. Fly by the seat of your pants (literally). How researchers can tap the plasticity of the brain to hack our 5 senses — and build a few new ones."

http://www.wired.com/wired/archive/15.04/esp.html

These experiments in creating new human senses have, to my knowledge, been restricted to the real world. But what better way to create new senses than through Virtual Reality, since everything in a Virtual World already has been measured and calculated.

What senses would you like to have? VR can give them to you. And the big question for me is this: do you get to keep your newfound superpowered senses when you take the Rift off?

[geekmaster]

"seeingwithsound" wrote:
... Thanks! Ah yes, some nostalgia. 🙂 You have a good memory, one of the snapshots of my old Compuserve website is at http://web.archive.org/web/20010123211300/http://ourworld.compuserve.com/homepages/Peter_Meijer/ (July 2000).

Hmm... I did not notice your avatar and username until just now. A little embarrassing to be pointing you to your own website, eh? It is an honor and a privilege to communicate (semi)directly with you! As I said in my linked post, I have been a fan of yours since the CompuServe days. 😄

[Bugnguts]

Touch is an incredibly dynamic sense which is why we have not been able to recreate very well.

Sight is simply a small fraction of the electromagnetic spectrum. We distinguish intensity, color and focal distance all received at two specific points on the body.

Hearing is sound waves traveling through mediums. The ear drums are sensitive volume and pitch and again only received at two specific points on the body.

Touch takes into account pressure on skin which is different from temperature sensory of hot and cold. But touch goes beyond simply the skin we can feel g's applied to our sub skin organs, hot and cold within our stomachs, lungs and veins. Then there is the feeling you get from your hair such as an ant crawling across your arm. There is also pain a burn feeling very different then a bruise or paper cut(not that most people likely want pain simulated). Summing it up touch includes the entire skin surface as well as internals and must not only be sensitive to pressure but also hot and cold.

To trick our sense of touch, as well as we have sight and hearing, will require much more effort and resources. I believe to simulate a human's sense of touch we will have to hijack the brain. Aside from that there may be some worth while avenues to explore but these approaches will be limited.

[jwilkins]

I think you oversimplify hearing. We are only sensitive to 3 wavelengths of light and we perceive many different spectrums of light to be the same "color" when they are actually completely different, but our ability to 1) distinguish frequencies and 2) phase differences in sound and to 3) distinguish infinite different combinations of frequencies (spectrums), and finally 4) the relatively low frequency and speed of sound actually makes it far more difficult to render sound than to render light.

We would have super vision if we got the same amount of information out of light as we get out of only single a pair of sound signals.

[geekmaster]

"jwilkins" wrote:
... We are only sensitive to 3 wavelengths of light and we perceive many different spectrums of light to be the same "color" when they are actually completely different
...
We would have super vision if we got the same amount of information out of light as we get out of only single a pair of sound signals.

There is evidence to support your claim about vision:

"

And there is evidence to support your claim about sound:

"At
Computer scientists have developed an algorithm that uses echolocation to build an accurate 3D repli...

Truly amazing stuff. Way beyond just hand and gesture tracking. Actually mapping out entire rooms with millimeter accuracy, and determining the user's postition within that room, just with sound. Awesome!

And even more extra awesome, like earlier studies on human echolocation, they believe us humans can be trained to echolocate like freaking bats! Unbelievable, but apparently true. ...

[Avon]

*Edit* Seems I was a bit off in my information below. It is not so much kinesthesia (bodily motion detections) so much as proprioception (bodily location detection) that is fooled by the mentioned illusion, though they are heavily connected). That does, however raise many questions as to what the brain is misreading in those illusions as there should be no reason for those senses to be affected. Perhaps it's a result of the way the mind reads data with vision or the immobility of the user is making the mind ignore that sense as there would seem to be no need to use it at that moment in time.

*Original post*

"everygamer" wrote:
The hardest senses to trick are going to be touch (us using our hands to feel objects) and Kinesthesia, which is the physical awareness of our own bodies. We might be able to in a VR environment allow someone to put their hands on a wall, and with bulky equipment, simulate your feeling touching that wall, but nothing is going to stop your physical hands from passing through the space where the virtual wall is. So even if we trick touch, we will not trick kinesthesia. The only way to trick Kinesthesia is to block our minds awareness of our body completely and replace it with false information.

I'm not too sure that kinesthesia is as hard to fool as it seems. The rubber hand and out of body illusion seem to imply that it's actually rather fragile and can be fooled if we involve enough other senses into the mix. Just brushing a rubber hand at the same time as the real one seems to work well enough, so I'd imagine employing full on sensory substitution or even light haptic feed back would be "good enough" to fool us in a Rift like fashion. The only real issue I foresee would be making that sensation last for long periods of time as well as with movement of the player as both the out of body illusion and rubber hand illusion have the user be stationary, a thing that just won't do in games, an interactive medium.

[360FOV]

I think technology is evolving much faster than anyone ever imagined even just 10 years ago. 10-20 years from now many things that we thought would never happen in a million years will be common place.

We have seen leaps and bounds in the artificial limb industry where neural impulses control robotic limbs. This was complete fantasy until recently and I'm convinced that once reverse engineered the same tech could be applied to creating feelings in the brain just like you are touching the object in real life... they already have this working on a small scale level.

The same knowledge will also be used to simulate any of the senses. The world we live in is hugely a world of interpretation happening in our brains and thus it can easily be fooled, so a few minor discoveries will make most of our VR dreams quite possible. The singularity is coming!

[JoshNYC]

"KBK" wrote:

Love it. Source?

[Avon]

"360FOV" wrote:
I think technology is evolving much faster than anyone ever imagined even just 10 years ago. 10-20 years from now many things that we thought would never happen in a million years will be common place.

We have seen leaps and bounds in the artificial limb industry where neural impulses control robotic limbs. This was complete fantasy until recently and I'm convinced that once reverse engineered the same tech could be applied to creating feelings in the brain just like you are touching the object in real life... they already have this working on a small scale level.

The same knowledge will also be used to simulate any of the senses. The world we live in is hugely a world of interpretation happening in our brains and thus it can easily be fooled, so a few minor discoveries will make most of our VR dreams quite possible. The singularity is coming!

To be honest, I have zero doubt that we'll have completely immersive virtual reality by the next decade (2020). The stars are practically aligning to provide the next generation of video game technology at that point, so this shouldn't seem to be too odd. Scientists are putting 100% photo real games to be released by 2020. Virtual reality technology's infancy is right now, 7 years before the "due date". Add in some technologies in the horizon (2014) that may provide for the day zero for virtual reality haptics, controls, and the increasing speeds of the internet, and the year 2020 seems like it's going to mark a turning point in history.

That being said, we must be careful where we tread, my group is predicting a firestorm of proportions the world has never seen before in that decade. Virtual prostitution, simulated drugs, transhumanist classism, disappearance of privacy, humanoid drones, increasing human workplace irrelevancy, political upheaval, water wars, to say there's a lot to deal with would be to put things gently. I'd say we should look to our current media in things like Deus Dx: human revolution, Sword Art Online, the Matrix and other such works to remind us that the future has many glorious possibilities that can just as easily fry our brains if we let them.

I suppose I've digressed far enough, back to sense hacking!

That very same sensory substitution mentioned in the first post could be used to help solve issues of motion sickness that are inflicting people at this moment in time. I'm currently investigating mitigation via the use of zero gravity chairs and haptics, but if a solution exists that can be implemented along side a full body haptic solution, may as well take two birds with one stone right? I'm still not too clear on how such mechanisms work however and I haven't seen a mention of cost or commercial availability at this moment in time so usage of such features may be a tool for v2.0 virtual reality set ups, something that won't be available for my group's v1.0 set up coming up late next year. If it turns out to not be expensive or particularly hard to implement, I see no reason why not to explore the potential in the near term to get more immersive experiences. Best start early to get ahead in the game. I want my VR MMORPG by 2022 at the latest, but I'll risk a lot more to shave of 2 or three years off that figure.

[360FOV]

"Avon" wrote:

That being said, we must be careful where we tread, my group is predicting a firestorm of proportions the world has never seen before in that decade. Virtual prostitution, simulated drugs, transhumanist classism, disappearance of privacy, humanoid drones, increasing human workplace irrelevancy, political upheaval, water wars, to say there's a lot to deal with would be to put things gently. I'd say we should look to our current media in things like Deus Dx: human revolution, Sword Art Online, the Matrix and other such works to remind us that the future has many glorious possibilities that can just as easily fry our brains if we let them.

Many of the things you have mentioned may be quite welcome among the masses. 😄 Unfortunately, disappearance of privacy, humanoid drones, increasing human workplace irrelevancy, and political upheaval are all going to arrive like a freight train and I don't think there is anything we can do to prevent it. If science can invent something then it will surely be utilized to its fullest even at the detriment of mankind. Nevertheless, I couldn't image being alive in a time where the most exciting invention was the cotton gin... no offense meant to Eli Whitney. Yet, it is an exciting time to be alive.

[Avon]

"360FOV" wrote:

Many of the things you have mentioned may be quite welcome among the masses. 😄 Unfortunately, disappearance of privacy, humanoid drones, increasing human workplace irrelevancy, and political upheaval are all going to arrive like a freight train and I don't think there is anything we can do to prevent it. If science can invent something then it will surely be utilized to its fullest even at the detriment of mankind. Nevertheless, I couldn't image being alive in a time where the most exciting invention was the cotton gin... no offense meant to Eli Whitney. Yet, it is an exciting time to be alive.

Definitely. While it'll be tough, there certainly won't be a shortage of things to do. The primary reason stuff like VR sex and simulated drugs concern me isn't so much on a practical level (who'd complain about these things?) so much as the legal situation developers like us will have to deal with during those times. Hot coffee was nightmarish for R*, so how bad would underage virtual sex be on the public. Would the government have to step in to regulate it as well as virtual drugs? How does one quantify rape or simulated torture in a legal system as it technically didn't actually happen in our "reality"? It's going to be a huge pain, but in all honesty, I welcome it. If that's what it takes to get the VR dream going than I'm getting my flame shield and rhetorical speeches ready ahead of time. All praise the VR revolution! 😄