What is the "Optimal" Virtual Reality Experience/Future

[Markystal]

I've been dancing around this question a bit in my VR musings and at this point, I think just plain old getting a discussion on the matter would be of some value. To start, I posit the question: "How do we want to experience virtual reality?" With gaming, movies, and other such media, we have a set of well defined, standard user interfacing systems. With games, people frequently sight the console gaming couch, gamepad, and TV. Movies have their theatres (home or cinema), and reading has the book and web article. So where does VR stand?

Right now, I see VR in a giant tornado of interfacing options, peripherals, and sci fi madness. Oculus has made their effort by positing the Oculus Rift as a seated experience, yet things like PrioVR, the Kinect, Leap Motion, or in other words, motion controls don't seem compatible with this line of thinking. I think we need to get things sorted out ahead of time right now before things get too out of hand and we risk damaging VR on the lines of motion controls, gaming's last big "innovation" that currently has the reputation of the Virtual Boy with "core" gamers.

In my opinion, I think the best route for virtual reality is a stationary, private system/pod/safe environment just for one user with a brain computer interface for preferably controls and immersion. The alternatives just don't add up to me.

Motion controls in the way we're currently trying to shoe horn into virtual reality and have been using in past years, seems counter intuitive. Think about it, what games work best with the Kinect? Typically, many people would say dance and fitness games right? Fundamentally, these are full body, ENCLOSED SPACE activities. Dancing is usually done indoors in club or party, a la the theme of most dance games, and fitness games tend to work best with their dance like or yoga based activities which yet again, don't require a lot of space. The enclosed space inherent to most optically based motion control systems don't mesh well with the very exploratory, high space necessitating gaming experiences most gamers crave (traversing, engaging with, and changing massive virtual worlds). Even room to room scale activities are too big for most people in real life. Newer, wearable based motion controls increase the mobility, but now bring into the fore that larger open spaces aren't infinite and will eventually bring hazards into the fore. With our heads in a virtual world, these can't be accounted for. Thus, I've concluded that motion controls are really best if left to the realm of Augmented Reality rather than virtual reality.

Game pads also don't sit well with me as a good alternative in the realm of controls as they tend to lend themselves to an almost ludonarrative dissonance-esque issue where our actions to control our avatar don't mesh with what we're doing. Even then, game pads are nowhere near capable of providing us controls that allow us to enjoy Virtual Reality for all it's capable of (god like workflow (literally), IRL replacment, sex, lightsaber battles, matrix, etc.) Not to mention that this would drastically limit the mass market appeal of VR as it's nowhere near as intuitive as things like touch screens, motion controls and BCI's can be.

As for the experiencing of the VR world, I think this is the one area where a surface brain/spinal cord stimulation is the only practical method possible that doesn't involve either surgery, matrix pods, or millions of nanobots. The rift and headphones are certainly good enough in my opinion for sight and sound, but the question remains on what we'll do for the rest of the senses (mechanoreception, smell, taste, thermoception, nocioception, proprioception, kinesthesia, etc.) No matter how much musing I do, the only thing I can think of in this region would be full body VR suit, but I think that people are troubled enough as it is just getting a rift and headphone on, let alone adding in things like and EEG, EOG, and now a full on outfit (likely a skin tight onesee with built in slippers and gloves...). I'm pretty sure scuba diving didn't become a popular pastime during it's giant Bioshock suit era.

Well, those are my overly long thoughts, it'd be great if we could get more input in on the matter. Perhaps people will open up to the idea of brain chips, implants and nanobots in the future and I'm just being to closed minded. Share the goods everyone. Glory on the path to the VR Paradise.

TL;DR
- How do you want to use VR in the future (BCI, HMD, Helmet, Brain chip, Hive-mind, cyberspace, mind upload, holodeck, anyhow, etc)
- Where do you want to use VR in the future (On the go, at home, private room, booth, VR cafe, life support pod, anywhere, etc)
- Who do you want to use VR in the future (set individuals, children, adults, licensed people, AI, families/communities together, anyone, etc)
- What do you want VR to be used for (Medicine, Entertainment, Education, Military, Productivity, anything)

[Markystal]

Astrocyte wrote:
What interests me is also how skills acquired in VR will translate into real world actions. Also, whether partaking in a VR education environment will be as effective as partaking in a real world one. I read recently that some people seem to absorb less info from reading things on a LCD screen relative to having real paper, but will this prove to be the case in VR, particularly if we are using OLED?

This prompt is composed of many variables that we'll need to take into account to be able to answer it. For one, at it's best, Virtual Reality should be indistinguishable from regular reality in every way. At that stage, going to a virtual classroom should only be different from the real thing in terms of tone, the nature of the world, and the capabilities of those involved. While it seems that kids looking at an LCD learrn lesss from those reading on paper, I have to wonder how much of that can be attributed to the subtle nuances in detail screens don't capture on a sensory level, like texture, matching the surrounding environments light (placing less of burden on the eyes to focus), and the lack of tactile feedback in scrolling through text rather than turning a page. Honestly, my biggest question at the point will be whether or not we're even using books and text as much in the future to teach kids.

Reading and writing, as well as math are are fundamental skills that you have to learn one way or another, but the question of whether or not reading will be the best method to achieve learning at that stage is equally relevant. Why would a history teacher of the future make kids read a text book or watch a video of what was happening in the 1700s when they could just VR teleport the classroom to a recreation of that time period. This can help in engaging the learners and could give students a much more tangible sense of what the subject matter they're learning does. I'd be ecstatic to have child of mine in the future (don't have any yet), tell me about how they went to the moon in VR and now want to try being an astronaut or how they enjoyed what they saw of engineering and now want to be an engineer. Quite frankly, it may be what's needed to spark some interest in the subjects among kids again as now a days (in the US and my own observations), most kids don't have much of an interest in science or math. I've noticed that kids increasingly want to know the "why" of things they're doing, like how many student's in highschool ask their Calculus teachers "When am I ever going to use this in real life?" The teacher could literally pop the answer up right in the middle of class and demonstrate clearly the function rather than trying to get the students to imagine why using an example (something that many kids that are already bored with a subject probably aren't paying enough attention to do anyway). In particular, I just can't wait to see VR allow kids to let their creativity run wild.

I'm really optimistic that VR can allow humanity to propagate and educate itself even more quickly than ever before. Imagine a VR equivalent of a how to video where you can not only see how something is done, but follow along right beside the person and do it right there (though admittedly this extends to AR as well).

The one issue that does extend to the practicality of VR however remains physical capablitites. VR is great as a method for teaching principles and techniques to users, but does pretty much nothing about their actual real world condition. I could train myself to the degree of skill of an MMA fighter in VR, but then again, that won't exactly help me out if my body's too weak to successfully execute the moves. While we can try, I don't think that VR will really be the best method of conditioning the body for specific IRL tasks (though it does have it's uses in training relexes, instincts, techniques and strategies). For that AR will probably be the king of the space.

Astrocyte wrote:
On hepatic feedback, check out this link to an article on Ultrahaptics!

Quite the interesting technology. Thanks for the link. If I had to say the biggest head ache that I have with VR right now, I'd without a doubt posit it to be the situation with the sense of touch. In order of importance, I place the sense of touch ahead of even our sense of vision. Yet, even at such importance, I don't think we have any one solution for touch that achieves the effects that we need. In a way, this sense is what will ultimately decide the final device(s) that we use the future of virtual reality. Oculus has given us a portal to another world, yet we're there without the ability to touch, as though some form of ghost intruding upon another world. I'll be going into a LOT more depth on the matter in another forum and a Youtube video later, but without a doubt, touch is the sense that has daunted me since the beginning of my inklings to pursue a career in virtual reality. Personally, I think my optimal VR touch solution would be some kind of wearable nerve stimulation device, though I can tolerate a small chair or pod. A las, it's the overall community's dream rig i'd like to know as my own standards for this aren't exactly average.

[ldevine]

I'll chime in here regarding the educational realm of VR and what many, myself included, are researching:

Creating virtual field trips is the obvious direction VR will go: recreate the Apollo Landing, witness the battle of Bunker Hill, and even explore the human circulatory system are scenarios that many educators are hoping to use VR to recreate, much like the Hololibrary. With today's tools, this will be pretty common.

What's even cooler is the possibility for collaborative problem solving in VR. Portal 2 is a great example of how two students can wear HMDs and work together to solve unique problems. This will have astounding effects of team development and leadership skills especially since students can create problem scenarios for each other to solve.

However, the UI as you all stated, is rather limited without haptic feedback. Many educational types of experiences will likely have very little interaction. I've been experimenting with the Leap Motion controller, and have found that it's biggest drawback, besides inaccurate finger tracking, is that many users get really tired, really fast. Holding your arm up to interact within educational explorations that take a few minutes, gets exhausting. We realized that we needed to give users time to rest, so we've had to limit the interactions to simple "point here" and "move there" gestures that only take a few seconds. The whole UX is in dire need of reinvention, because how a student interacts and engages with the material greatly contributes to how they learn; passive learning doesn't work. Joshua Kim made an astute point in his article Avoiding Oculus VR Education Fantasies, realizing that:

"Education requires a personal relationship between educators and learners. Education requires that the learner produce, not simply consume or experience."

http://www.insidehighered.com/blogs/technology-and-learning/avoiding-oculus-vr-education-fantasies#s...

Without a teacher like figure, education tends to move at a slower pace, and that's where things get really interesting...

We have been researching Intelligent Tutoring Systems with Virtual Human Tutors that students can converse with. Virtual Tutors have shown some real progress as supplemental experiences to in-class education as they guide students through science explorations with animations and interactive simulations, all through conversational dialogs. Through dialog, the tutor can produce questions that challenge your understanding of the presented concepts, assess your verbal response and then respond accordingly by presenting further materials. Despite their 2D interaction, one of the most surprising outcomes was that young students have expressed a developing relationship with the tutors that helped motivate them to learn more. So with VR, we are hoping to see significant gains on students developing stronger bonds with Virtual Humans and furthering the educational experience because a student can have a sense of presence as they sit next to, or walk beside, their tutor.

At least, that's one direction we;re headed.

[Astrocyte]

Sorry it’s taken a while to respond, works been busy.

"Markystal" wrote:
.Virtual Reality should be indistinguishable from regular reality in every way. At that stage, going to a virtual classroom should only be different from the real thing in terms of tone, the nature of the world, and the capabilities of those involved.

Completely agree, although given the complexities involved I think it will be some time before we get to that stage. Still, that may be just as well as we need to be cautious given VRs potential. For nostalgia I watched “The Lawnmower man” the other day. Let’s just say the film has not dated well! Even though the concept of VR has been around for a long while now, it kind of feels like this is really it this time!

"Markystal" wrote:
While it seems that kids looking at an LCD learn less from those reading on paper, I have to wonder how much of that can be attributed to the subtle nuances in detail screens don't capture on a sensory level, like texture, matching the surrounding environments light (placing less of burden on the eyes to focus), and the lack of tactile feedback in scrolling through text rather than turning a page. Honestly, my biggest question at the point will be whether or not we're even using books and text as much in the future to teach kids.

Spot on, although I think screen flicker also plays a role here even if its below conscious detection. Great article in Scientific American on the topic of learning and displays:
http://www.scientificamerican.com/article/reading-paper-screens/ Also, see research article from last year which reported that students who read texts in print scored significantly better on a reading comprehension test than students who read the texts digitally.
http://www.kau.se/sites/default/files/Dokument/event/2012/12/mangen_a_2013_reading_linear_texts_on_p... .
To take your point about the history teacher, sure, you could just VR teleport the classroom to a recreation of that time period. However, as I have an active imagination, I get this by reading also 🙂 . VR could also help relay content to those with a specific learning difficulty such as dyslexia, especially as may dyslexics are visual people. But, I think that written works will always have some appeal, and for a sizable portion of people a preferred option (e.g. those who can’t see stereoscopic images for whatever reason). I agree though that VR could be used to spark interest in the science subjects among kids (AR could also do this by real-time animation of class science practical’s). I agree that VR could be used to boost creativity, particularly if it becomes a ubiquitous technology. However, there are two bottlenecks to be negotiated here before that happens. The first relates to hardware costs. Although the Oculus will run on most modern PCs with a reasonably good GPU, one still likely will need a good spec gaming PC in order to get the most out of the rift, and is an expense many people cant afford. Hardware costs could however, be brought down if the processing power needed to render high resolution graphics at high frame rates was moved onto the rift itself, taking the demand off the PC to some extent. Until this happens widespread adoption is likely to be limited. Also, I am fully expecting cost hikes for higher end hardware when CV1 is officially announced. The second barrier relates to the costs/restrictions associated with producing VR content. Currently only URE4 is within the price range of most people interested in getting into developing, and even then the 5% on the retail of any games produced with it will certainly hurt the profit margin for many would be developers. This and the outright extortionate costs of using other commercial gaming engines will keep the costs of VR content high for the foreseeable in my opinion. Many people would benefit from open source (more cost efficient) alternatives that are as good or even better than URE4. Not holding my breath for that though! Also, these applications need to be more specialized. My cognitive neuroscience research would be helped greatly for example, by using Psychology experiment presentation software (like OpenSesame) that was built specifically for VR from the bottom up. We need a whole range of specialized applications in order to be fully creative with VR because its potential extends so far beyond just gaming.

"Markystal" wrote:
VR is great as a method for teaching principles and techniques to users, but does pretty much nothing about their actual real world condition.

Whilst just imagining doing a particular activity does enhance real world ability to some degree, the lack of feedback from the muscles does limit the benefit. But, this will likely to change when hepatic feedback is capable of closing the loop that so there is true two way communication between the simulation apparatus and a person’s body. I look forward to your Youtube video on VR and touch!

ldevine, thankyou for your insightful input regarding the educational realm of VR.

"ldevine" wrote:
Portal 2 is a great example of how two students can wear HMDs and work together to solve unique problems. This will have astounding effects of team development and leadership skills especially since students can create problem scenarios for each other to solve.

I'm sure it will, and this point was missed by Joshua Kim I think, who it seemed to me couldn't see beyond VR as just providing a gaming experience. Your spot on with the haptic feedback issues, and I can see how people would tire quickly with the leap motion controller! I also agree that without a teacher like figure, education tends to move at a slower pace, although this is partly because children are not generally taught the skills necessary for independent learning. Not sure what age the students are that you are referring to, but I’m assuming that we are talking about children and young adolescents rather than older students. Although I am very interested in your research, I am not yet convinced that Intelligent Tutoring Systems with Virtual Human Tutors are better than real ones at present. Firstly, do the students know that the tutor is not real? Beyond constrained specific circumstances, I find it hard to see how these virtual humans could be anything as nuanced as a real tutor when things get complex (how many time s for example did a real tutor have to step in to correct a tutor that had crashed or was churning out something that was not making sense to a student?). Whereas young children often accept at face value what they are presented with, no adult human has yet been reliably fooled by a conversation with a computer for long. Also, in view of the known issues around the reduced comprehension of students reading from a screen vs real text, were there appropriate controls in your experiment against students receiving instruction from real tutors/real books? Besides educational content, young children are also learning about important social skills from their teachers, and I’d be a little concerned about an over-reliance on computer aided learning as a potential replacement in this context (I say replacement because some will seek to reduce the number of real tutors in favor of virtual ones to make savings from the educational budget). Since computers are currently non-sentient, this poses some dilemmas in my view. So you can see why I am skeptical about young students forming stronger bonds with Virtual Humans in VR. We don’t want to be improving grades at the expense of social skills and empathy! Although I think your research has real potential as an assistive technology, I am concerned about the impact in other areas central to most human lives. Still, I am open to changing my mind and I look forward to hearing more about your project and in particular, reading the peer reviewed report!

Thank you both for such stimulating discussion! 😄

[Adder]

A three meter sphere, sitting in a matching cage (2/3 buried in the ground, or steel supports in a trailer) like a ball bearing cage. The thing could move all directions, and have some mechanised rollers onto the underside of the sphere to stop it spinning too quick or drive it slightly in certain directions of spin.

The user gets in the hatch from the sphere's (slightly weighted) upright position, standing with the wireless HMD powered by a small battery backpack w/ attached cushion for sitting down.

A pair of haptic gloves to simulate joystick handling, keyboard feel and scenario interaction.

I think that would be enough for a really good pair of gloves and HMD set to create a persistent immersion.

[Astrocyte]

Interesting idea adder, reminds me of something out of the film 'The lawnmower Man'. I'm not sure that this would be easy for most people to accommodate and use. Also, the Oculus is not likely to be wireless for a while yet due to limitations in the necessary hardware. Major improvements in battery technology have to occur otherwise one will be charging the thing frequently, which would be annoying.

[Markystal]

"Adder" wrote:
A three meter sphere, sitting in a matching cage (2/3 buried in the ground, or steel supports in a trailer) like a ball bearing cage. The thing could move all directions, and have some mechanised rollers onto the underside of the sphere to stop it spinning too quick or drive it slightly in certain directions of spin.

The user gets in the hatch from the sphere's (slightly weighted) upright position, standing with the wireless HMD powered by a small battery backpack w/ attached cushion for sitting down.

A pair of haptic gloves to simulate joystick handling, keyboard feel and scenario interaction.

I think that would be enough for a really good pair of gloves and HMD set to create a persistent immersion.

Woah, I supposed some people are more willing to accept larger interfaces than I expected. Then again, my own VR rig design for this year does suggest an anti-gravity chair (those can certainly be rather large). In this situation, I agree with Astrocyte that a wireless Rift will likely need to be developed and the battery power needed to keep the thing running for enough time will likely be a problem. While I personally don't like the idea of having to move much at all for my VR experience, if one does go with that route, might I suggest looking into and outer lining with the pulsed sound sensation technology to provide haptics and a pair of wireless headphones to immerse more the senses (alternatively a speaker system could also work here since a rig this large likely doesn't need to concern itself with keeping the experience too isolated.

Fascinating information Idevine. I agree that haptics and control methods are one of the areas where virtual reality is still currently lacking. Having a system that provides solutions for these without the need for a lot of physical exertion is one of the things I believe VR will need to solve before it can truly thrive. On the note of the virtual tutors you mentioned, I'm actually equally as curious on using such learning to facilitate AI development. One approach to AI currently being taken is to create a basic AI that learns as it goes, kind of like a child. This would allow the AI to get more information on how to interact with humans and steadily improve peoples acceptance of AI, a social change that I think may become more necessary as machines and computers are only becoming a more and more prominent part of our daily lives.

[itcamefrommars]

There are more options for experiences with VR... I have to disagree with the stationary is best. I've had people hug me and say we've changed their lives with our mobile rig. No kidding haha... It's pure magic... but there's a lot of personal interaction involved also.
http://www.reddit.com/r/gaming/comments/23efx5/oculus_rift_bourbon_st_edition/

anyhow we are actively exploring and experimenting with different ways of using VR tech as parts of experiences... not as the whole experience.

This is a very important and emerging question you bring up. I will try to get more information on what we are doing up soon.

[needsloomis]

Right now the process of development is to simply solve a VR problem, and see what new problems arise. An example would be the first time we all put on our rifts, looked down, and had a minor shock when we didnt see our hands/bodies. Then STEM came around, but with its own issues. Ad infinitum.

Keep in mind, it also took years of development to make the rift, and decades of tech to surpasses the gigantic silly amusement park headsets. If you told me in the 80s that you suspect VR will be possible in the late 2000s because the successor to the satellite phone will drive the markets towards the appropriate technology, I would consider your ideas far fetched.

So what Im getting at is how it will work depends on the unique problems we discover as we go, and what technologies will be available in the future, and both of those are giant unknowns.

What we want out of VR, however, is simple. We want to strap into our VR devices, walk into a world indistinguishable from reality, and bang hot alien chicks.

[kressdev]

Currently TMS seems to be the most effective way to get immediate real results. For example, Google "Illusory Sensation of Movement Induced by Repetitive Transcranial Magnetic Stimulation" by Christensen et al. 2010.

Today, I think TMS is still too crude to be used in VR. I think it would require additional time to develop a more sophisticated version, not resembling the current commercial devices. You will likely also need a monitoring/imaging system working alongside TMS (eg. fNIR, possibly ultrasound, etc.).

Alternatives include thermoacoustic tomography (infrared or microwave), although I am not aware of this being used transcranially for stimulation, only imaging. Also this significantly heats the tissue...not good.

There is one recent study using transcranial ultrasound (Legon et al. 2014), but none of the volunteers actually felt any effects of the stimulation.

[cerebral]

An augmented jurassic reality.
You buy some land in New Zealand, Brazil, or wherever and let people with AR goggles and a potent battery powered
PCs in the backpack walk through the jungle populated with virtual dinosaurs that have also an AI and could basically virtually live there.
To prevent the awfulness of visual CG failures, for example if a dinosaur walks through high grass,but the grass wouldnt move a centimeter you could cut out some paths from the environment and replace it with virtual plants that have collision detection.
A simple AI would be really interesting because for every tourist it could be then an individual experience if a T-rex reacts everytime differently to ones movements.
A backpack could weigh sth around 6kg including battery(be charged at night,replaced at various locations). You could do camping in the jungle, run away from dinosaurs (survival game) etc...possibilities are endless, find other humans to survive.
Or an analogy with zombies in an abandonned city. :mrgreen:
I hope VR headsets will get rapidly better and rendering capabilites will augment with eye tracking because future might be a lot of fun. :lol: