Waiting is hard... (Ideas while we wait)

I like this idea. To take it further, imagine an entire digitally scanned live environment ( a party, for instance). You can walk around in it, fast forward, rewind, pause, explore in great detail. Sure it's still years away, but I guess I can wait. :)

This would also translate so well to AR. Have that person there in your room with you and your friends, whether it's a celebrity dancing in the middle of the room, an inspirational figure giving a speech, a friend miles away, a loved one long gone or a youtube figure making commentary... Or all at the same time, with 3D audio, letting you focus on one person at a time if you wish.

Virtual Teleportation.

With surprise personal virtual visits to your own room by celebrities, that you can have a conversation with. It's like the invention of photo, telephone, network broadcasting, home video, on demand content, personal computers, internet, cgi, 3d multiplayer games, social media, video blogging all roled into one big conclusion: We will never be apart again.

id tech 6 research type ray casting and voxels used for data compression, using hardware acceleration with modern special units and low level access, combined with streaming of large data as needed a la id tech 5 megatextures, depending on where you are looking with the head tracking data the rift provides, probably with eye tracking in the future, made to be a solid fps experience with scalable graphics quality per frame, again a la id tech 5, using a foveated approach when possible, timewarped and everything for a smoother experience...

Real time 4D broadcasting would be the goal, as virtual teleportation. But bandwidth would make it hard for the first generation. A PC with high-end SSD, multicore heavy CPU and multi GPU graphics with low level API access should make it possible for the precaptured material to be displayed locally. Maybe merged with CGI background, maybe in a real time rendered environment, or a hybrid solution.

A 1157 horror experience / Marilyn Manson video / Quake2 Strogg torture chambers combination type VR horror experience, in UE4 archvis type spaces rendered in real time, with 3D 360 degree skybox type horizon captured from real life, TV's that look like security monitors from other rooms displaying 2D content, with lower quality rendered 4D captures the way Technolust already does it, looped far from the viewer, with real time rendered events, happening at a distance you cannot tell the difference in quality, and real life captured characters next in line for what's happening over there, in this "Hostel" type environment, blindfolded and gagged, tied up to chairs, next to you crying and screaming... (No need for their eyes to track you, or them to respond to you, or have any type of AI). With this guy in a gas mask, walking around in volumetric fog that moves around him, deciding who to pick next, getting in your personal space, maybe asking you questions that you respond to with your head, and your answers determine what combination of these 4D captures is played next... Now in this case, it is you that is virtually teleported.

Dressing rooms with mounted 180 degree 3D cams and a 4K GearVR. You wanna know if that dress makes your butt look fat... turn around and see for yourself.

Well, you would not want cams in the dressing room itself, so it's just a room next to it. With a mirror facing the cam at an angle allowing you to see front and back on GearVR and to take the GearVR off and check yourself again in the mirror.

Shopping AND an out of body experience :)

E3 2015 will be something else.

___________________________________________________


Display tech on the desktop space is moving fast
1. RPS guys seem to be happy with current 40 inch 4K monitors for gaming.
2. 144Hz IPS G-sync monitors were unveiled at CES.
3. HDR support has been added to HDMI 2.0a spec
4. LG has 8K monitors on the way

2016-2017, we'll probably be able to get a 32/40 inch 4K/8K HDR 144hz G-Sync monitor/TV for the desktop, maybe requiring a dual cable setup or whatever... It will be instant eyegasm.

___________________________________________________

GearVR will be 4K this year. When viewing movies, the virtual movie screen will have a resolution of 720p. The Rift may not be so different. This is barely enough to not look crappy compared to the current standard in TV/monitors. Just like with CB, we'll here comments about how it's not as crisp as a desktop monitor but it's cool. Even to keep that relative image quality of "meh for some, cool for others" requires yearly resolution bumps to keep up with the trends in monitors.

Thankfully the display tech on the mobile space is also moving fast.
___________________________________________________

DK1 @720p, late 2012,
DK2 @1080p, mid 2014, uses Note3 screen
GearVR Note4 @2K, end of 2014, CB @1K per eye mid 2015?
GearVR Note5 @4K, end of 2015, CV1 @2K per eye end of 2015 - mid 2016?
GearVR Note6 @8K, end of 2016, Rift @4K per eye end of 2016 - mid 2017?
CV3 @8K per eye, end of 2017- mid 2018
CV4 @16K per eye, end of 2018- mid 2019
___________________________________________________

In just a few years, VR may reach 16K per eye, which seems to be considered the holy grail. It may just have to get there to be acceptable compared to monitors and the competition in the mobile area may help. By then, the input solution, tracking, tools for content creation, the app market and platform will all be there. GPUs with lots of stacked memory at high bandwidth and low level access by programmers will be the norm. Developers will have a lot of experience creating VR games and experiences and great examples and guidelines by Valve and Oculus.

We'll also see big movie studios opening gaming studios, or merging with some, bringing experience and know-how from all kinds of computer graphics and content creation backgrounds together to create the new Hollywood. They will have products like movies in VR format using 4D captures, 4D CGI and real-time rendering for an unprecedented experience. Imagine Animatrix, Matrix Reloaded and the game that fills in the gap but instead of watching the movies and playing the game separately, you are in the VR movie/game. The actors talk to you, just like they are there.

Insurgent demo looks and feels fantastic for now. That will change with 4D video captures in VR become the norm. 3D video based humans in VR look like 2D spite effects in 3D games. Your brain just doesn't buy it. 4D captures in VR spaces at 16K per eye will be hard to not buy.

___________________________________________________

Waiting is hard. Partly because atm, Oculus knows it's not me they have to dazzle with the next big thing in store and get high on their magic future tech. It's the big names. Like multi-billion dollar names... Whoever makes and delivers what creations billions will be looking at in the future.

But on a good note, if the roadmap I made is not so far off, a CB @1K per eye can be released mid 2015, which is the perfect time for an input devkit, because it is when E3 takes place, where a demo can be showcased to the public using all these components.

An input devkit, that would come with that E3 demo, that you can get for your DK1 or DK2, or with a CB if you wish, will be here this summer. The CB option would make this a limited test run for Oculus for the two display based model, and the start of the production of components that will be used in CV1 with upgraded components like the display.

Carmack said he'll be spending more time on software at Oculus. Palmer said in-house made exclusive content will be there. Carmack is great at being able to put together something that would be "more than words" and knows that's what is needed to push VR. They have a E3 demo floor where they can just murder everyone with in house made content for gamers, which is the best thing the guy has been known for his whole life, using their input solution that can be ordered online at the time, and a CB. They can just murder and still say "btw, the CV1 will be this, but with better resolution".

What they show there will probably be the first public demo of an IP that will be their flagship.

___________________________________________________

This just lines up with all the confirmed and unconfirmed clues we have had until now:

design for CV1 being pretty much locked down,
that design following CB,
CB being a two display solution,
CB rumored to be @1k per eye,
The Vive rumored to be 1k per eye for dev kits,
talks of a release of an input kit for devs,
talks of limited summer release of a Rift based on the CB design,
The Vive rumored to be 2k per eye for the consumer version,
a Note 5 level display manufacturing tech enabling panels for CV1 at 2k per eye,
Oculus-Samsung product cycle so far,
Carmack at work on software at Oculus,
Palmer talking about exclusivity of in-house made content,
the upcoming E3,
Samsung bringing it's full PR power to the table...
Oculus VR filing for Crescent Bay trademark

___________________________________________________


E3 2015 will be something else.

1. Rendering passes and parts of the moving world where the eye is more sensitive to temporal resolution, rendered at full speed, at half resolution
2. Parts of the 3D world where the eye is more sensitive to detail , rendered at half speed, full resolution, timewarped
3. Layered on top of each other with a fast pass of noise on Z data

If layering in the next SDK is as flexible as that, or is intended to get to be in the end, we can see some cool stuff.

It could also be cool to have a very low resolution flexible screen panels at twice the frame rate of the main display, located in the hmd around the lenses, used to provide ambient lighting beyond the displays fov. Could simply interpolate ambient color at 2x fps, creating another layer, in hardware.

Why can't this on your wrist be the input solution:



Having the fingers at a fixed distance should make the tracking go a lot smoother. Could also work for scanning the controller you are holding.

Could have IR leds and sensors on it, just like the Rift. Add two fixed cams instead of one, placed around the room similar to LightHouse stations, it should enable hand tracking, finger tracking, 360 degree head tracking, controller scaning and everything.

Could also have those "emotion sensors", heartbeat and everything if that's necessary. In games, that could be where your notifications are displayed, like a virtual smartwatch. The one on the left hand could display your webcam's feed and get linked to your mobile's notifications, bringing your real world inside the virtual one.

________________________________________________________________________________________


A bluetooth capable version with a battery could be used with Gear VR with other IR cams on the Gear VR to track the IR leds on the wristband for hand position tracking, relative to the head. And two self powered IR led devices to place around you wherever you go for the Gear VR to track its own position in space relative to those, with the same IR cams on it. The IR led devices are fixed in space, so by knowing the head's relative position changes, you know the absolute position of the head in space, similarly you know the absolute position of the hands in space. No cables.

So the whole set is:



Has 3 more IR cams at the backplate, using CB design with nice weight distribution. The cams having 120 degrees of fov each, there is no blind spot. Tracking 360 degrees around you.

Would that not work?

Optional VR runner pack upgrade:



Two bigger versions of those wristbands, that go on your ankles, with 3 IR cams to cover 360 degrees. One more IR led station, that goes on the floor.

That should work for movement. The sensors would work to detect rotation and movement speed, as you run in place. The IR cams on the anklebands, detecting their position relative to the stationary IR led units would be used for positional tracking, feeding the data wirelessly to your GearVR. No drift.

You would not even need to lift your feet from the ground. Just moving them would be enough. You could also be sitting on a chair doing that if the seated experience is a must. It would also work with an omni, a sphere, a virtual bike, a treadmill...

________________________________________________________________________________________

Even the basic package would provide positional tracking for the head in cable-free portable VR and enable good enough hand tracking at least when the hands are in your view. Wristbands could be an optional upgrade package and so could the anklebands with an additional IR led unit.



________________________________________________________________________________________



Complete package:



What would potentially make this whole package work flawlessly (in terms of performance) is a SOC on GearVR, getting all that data from the sensors and IR cams on the HMD, wristbands and anklebands, processing it and sending only the final positional data to the phone. Updating head position data more frequently than hands, hands more frequently than feet, keeping latency at minimum for the head. So all that the SOC on the phone does is rendering, timewarp and predistortion. That could potentially help with overheating too.

This complete package with a SOC might sound like too much, but I have a feeling that all this would not cost more than the Vive with wireless controllers and LightHouse stations. It should also be fairly user friendly as everything is battery powered and you can just place the stationary units anywhere, with one of the IR led units on the floor, followed by the "reset all positions gesture" and it should work.

________________________________________________________________________________________


CV1 for the PC could be the same package with the HMD having it's own displays and a single cable connection to your PC, not needing a battery, with the stationary IR led units also with cables instead of batteries, bringing cost down. The PC could still just do rendering, timewarp and predistortion, providing more eye candy and bigger experiences than portable VR. The SOC on the HMD could also be dropped for cost reasons, letting the PC crunch positional tracking data, just the way it works now.

Stacked memory async multi GPUs with low level access APIs must be as big a generational leap as X360 to XboxOne. It is the next gen and it's here this summer.

I hope someone (working with an experienced manufacturer like Asus) makes it into a wearable VR PC, with a battery packing notebook-like design, minus the screen, keyboard and touchpad, plus much better cooling, straps and padding between it and your back . That would make untethered VR for the PC possible, without the need for a wireless display solution.



Could pack additional sensors and IR leds too. Combined with the set above, you'd have untethered VR, with next-gen quality graphics and almost full body tracking, with hands, feet, fingers and their relative position from the body.

This "IR cam on the hmd, IR leds on portable units" approach could be the solution to Oculus not being laser sharp focused on a single product now (as stated by Carmack), by unifying the tracking approach and design for both PC and mobile and making untethered VR with positional tracking possible on both platforms.

It would even be possible to unify the product lines, with the last piece of the puzzle: A Galaxy Note (or S6) sized device packing dual display panels + displayport connection to a PC port and USB pass through connection to GearVR to connect it to a PC (no battery, no SOC, thus lighter and cheaper to produce), something that can be inserted to GearVR, making it compatible with a PC. Just one combined, replaceable cable from the wearable PC to the Oculus Display. It would not only unify the Oculus lines but also make it easier to upgrade the Rift. It could also make us, the fans stop worrying about the mobile market getting all the loving and the PC getting the short end of the stick as the products would be one and the same.



The pieces are all there for an ecosystem that lets users choose how deep they want to go into the rabbit hole (and their pockets) by offering compatible products that come together:

1. GearVR with IR cams and two portable IR led units
2. Galaxy note, OR, a same sized dual display + connections pack to connect to a PC
3. Wristbands
4. Anklebands with one IR led unit
5. Wearable PC

To bring the same experiences with scaled down graphics on mobile VR for the masses and this visual quality to the enthousiast market with high-end wearable PC VR:



It would be like Oculus launching the Walkman/iPod and the Surround system that plugs to it.

"In an ideal world", what would enable this on the software side, could be two Carmack made renderers, dropped in UE4. One designed and optimized for fixed hardware on the mobile VR side, one based on Vulkan, designed and optimized for the fixed hardware in that wearable PC.

That would make perfect sense if you think about how UE has user friendly tools, is free, has the source code out there, has the renderer running in its own thread, has different builds of the renderer dropped in by for example nvidia with their version supporting VXGI, how even Quake2 had the renderer in a separate dll and how Carmack said the renderer in Rage was not the hard part that took much of his time, it was the rest of the engine.

I get excited thinking about all these possibilities, then bummed out remembering that "the key to happiness is reasonable expectations".

I feel like Cartman having to freeze himself till the launch of the Wii here, waiting for "the dawn of home VR"... Meanwhile, HL2 Update on DK2 is beyond my childhood dreams.

And as we take our loved ones to be scanned so we get to keep their memories forever

This is actually one aspect of VR that worries me a little.

If we get to a stage where loved ones can be realistically scanned and their avatars given a certain degree of simple bahaviours/actions (perhaps even recordings of their speech) such an application might become an unhealthy 'crutch' in the event of bereavement.

A person might miss their loved one so much that they become reliant on the false virtual world in which a facsimile of the person still exists and this would not be psychologically healthy.
(The grieving process generally needs to go through certain natural stages in order for an individual to accept the loss - this is one of the reason I have a real issue with 'psychics' as their actions interrupt/delay/derail this process with false stories and the creation of memories that actually have nothing to do with the real departed person).

If I lost someone close to me and I knew I could put on a headset and see them sitting next to me again, smiling, talking... it might be very hard not to become very dependent on this.

I find the risks associated with this to be very similar to the ones for cannabis. One is emotional dependency and addiction by emotional dependency. The other one is how it can affect the perception of reality at a young age where the brain is still developing. I think we will see a lot of controversy centered around these two. I would advocate the legalization of cannabis but with an age limit, so it's hard for me to say VR would be safe for children. You get haptics in there, the father/mother who past away looks real, feels real and the emotional impact would be hard to master. That could also take in-app purchases to a whole new level.

On the other hand, this will open up so many possibilities for therapy. Guided VR sessions with pharmaceuticals. It might be a healthy way to say good bye to loved ones, under supervision from a pro. Or to overcome a bad memory, replacing it with a virtual one with repeated sessions. Honestly, that's gonna make todays therapy sessions look like a joke. On the flip side, brainwashing will get to a whole new level.