Forum Discussion
Sharing John Carmack's advice for using 3D panoramas as game environments (e.g. Defense Grid 2)
When discussing with John about the implications of his current research on RGB-depth 360 panoramas in mobile VR (TL;DR: perf cost to render is too high), he provided the ff. advice for implementing an advanced optimization technique for achieving the visuals seen in OTOY's synthetic stereo 360 photos in a game context:
A similar optimization can be found in Hidden Path Entertainment's Gear VR port of Defense Grid 2. AFAIK it's the only published title on any VR platform to even attempt such a technique. While the results are mixed, the decision to pursue this technique should be highly praised from a developer's standpoint, and its value to the dev community cannot be overstated.
You can bake view dependent offline rendering onto crude geometric approximations to improve on the simple sphere or cube of standard stereo 360s.
(link to Tweet)A similar optimization can be found in Hidden Path Entertainment's Gear VR port of Defense Grid 2. AFAIK it's the only published title on any VR platform to even attempt such a technique. While the results are mixed, the decision to pursue this technique should be highly praised from a developer's standpoint, and its value to the dev community cannot be overstated.
4 Replies
- It's a great tweet. Really gets the creative juices flowing.
Would be interesting to see a video version of this technique. I imagine that would involve assembling a monoscopic 360 video of an on-rails moving game environment (or one where the camera is moving through it), that then gets projected onto simple moving geometry that's synced up with the video. Would only need to play standard equirect videos instead of the giant square videos with stereoscopic views baked in, and would only require half the storage space? And it would no longer feel terrible if the player tilts their head vs the horizon! - If you want to play with equirect + depth to create 6DoF video, check out my free app, Pseudoscience 6DoF Viewer, available for Windows MR, Vive, and non-vr Mac and PC (also an Alpha build for Rift. PM me if you want in on the Alpha). Here's a demo video: http://youtu.be/1tBS66cWBno
Links to downloads and a variety of 6DoF photos and videos are available at reddit.com/r/6DoF - That is very interesting stuff, Pseudoscience! I didn't realize any VR cameras were producing depth maps in their output. :open_mouth:
- Yes! Google Jump (so GoPro Odyssey and Yi Halo), the Kandao Obsidian, and Nokia Ozo all provide depth-maps out of the box. I've also released a free Python app for calculating depth maps and converting stereo 360 footage to 6DoF, so a lot of VR filmmakers and photographers can get their stuff into 6DoF, right now!
The Stereo2Depth software can also be found at reddit.com/r/6DoF.
Here is a demo video that shows the full workflow, from downloading stereo 360 footage from YouTube, to converting to 6DoF, to viewing in VR: youtu.be/i-QTbpjj4Mw?t=41
