Current best quality video playback on DK2, Gear & Cardboard

I think GearVR with a Note 4 is the best for 3d video generally. Sometimes the DK2 looks better with 2d 360 videos (because its wider angle of view than the GearVR makes the 2d 360 video look less flat -- but the screen door pixel visibility is much worse on the DK2). All these panoramic 2d and 3d videos will look a lot better when there are HMDs with much wider FOVs. This is the critical thing for videos even more than games I think.

I forget where I gathered this info. Please correct if any of it is wrong.

H.264 limits of level 5.1
3048x3048 @ 25 FPS
2048x2048 @ 60 FPS

H.264 limits of level 5.2
4096x4096@30
2968x2968@60

Which gives a maximum of 504 MPix per second
Increasing the frame size will decrease the frame rate and vise verse.

H.265 limits of Level 6.1 - Currently limited hardware decoders support and patents are preventing it from being a usable solution.
5944x5944@60

GearVR S6 limit
2160x2160 @ 60 FPS
6 pix/°

DK2 has a ~94°x106° FoV and 960x1080 pixels per eye
CV1 has a ~94°x106° FoV and 1080x1200 pixels per eye
HTC Vive has a ~104°x116° FoV and 1080x1200 pixels per eye

Need 4160 pixels per 360° to get a roughly 1:1 pixel mapping on CV1. But even with the DK2 and GearVR image quality is improved with making panoramas with cubemap of 1536 for 90°. I think the oversampling is used to help give a better image on the screen as you move your head. That works out to be about 4800 pix per 360° for the DK2, which gives us 5400 pixels for CV1.

4800 pix per 360° = 13.3 pix/°

Playback limitations result in resolutions 1/2 the width and 1/2 the height of what looks best on the screen.

Until I upgrade my PC I can play larger movies on GearVR than on my DK2.

Thanks for the numbers.

I think some of these change some with bit rate settings override.

Thanks, j1vvy...great info.

Does anyone have any experience with OTOY's ORBX cubestrip of 18432x1536? Their support team tells me that video is supported in this format on the Gear VR, but I can't imagine how playback is possible at any useable frame rate.
And I don't think I can build an MP4 of that resolution, even with ffmpeg.

"redparis" wrote:
Thanks, j1vvy...great info.

Does anyone have any experience with OTOY's ORBX cubestrip of 18432x1536? Their support team tells me that video is supported in this format on the Gear VR, but I can't imagine how playback is possible at any useable frame rate.
And I don't think I can build an MP4 of that resolution, even with ffmpeg.

We hit a wall with MP4/HEVC. The HW decode buffer tops out at 3840x2160@30 fps on Gear VR, and isn't much better on PC.

To get 18K cube map videos at 60+ fps we went in a different direction. We compress the 18432x1536 frames in a device specific format (.okx - exporter is up and running on orc.otoy.com, coming in Octane 3 ). This data can be decoded on a GPU in pure OpenGL, and doesn't use the CPU or HW video decoder. The ORBX container which packs the data adds another layer of compression getting us another 20-75% depending on how much noise/motion is in the scene.

Right now, on our Gear VR app (Note4+), we can load 10-24 second 18K ORBX videos and play back at 60+ fps. We are working on supporting this quality as a continuous stream from local storage with newer phones. The next step after that is WAN streaming through X.IO.

On PC, we have 18K stereo cube map videos decoding at ~120+ fps on a GTX 980. This is seeking/streaming right from disk with no buffering at all, meaning we already support video and navigable volumes of infinite sizes at this res/quality, limited only by local storage size.

The 16K @60 fps video wall/movie player at our Siggraph 2015 booth was running an earlier version of the gigapixel GPU video decoder:

https://twitter.com/otoy/status/643469086819618816

This is all an interim step towards volumetric/light field media (both rendered and captured), which is a going to be basic requirement on HMDs with position tracking.

Looking forward to the Gear VR motion version.

1. Will OTOY make a converter so other media besides render outputs from Octane 3 can be used? (i.e. video)
2. Any chance of using your GPU compressor on other media?

"scottsquires" wrote:
Looking forward to the Gear VR motion version.

1. Will OTOY make a converter so other media besides render outputs from Octane 3 can be used? (i.e. video)
2. Any chance of using your GPU compressor on other media?

1) yes, although the converter will probably leverage Octane.exe+lua script in command line mode to do this with aribtrary input files. Octane is still the best way to convert spherical to cube map, even if it is just video frames.

2) yes, it will work on video or captured photos/videos too.

Looking forward to it as you release info and updates.

is there a SDK or framework which one can use to build a custom Video Player ?

"THINKINGZOMBIE" wrote:
is there a SDK or framework which one can use to build a custom Video Player ?

After we are done with the 3.0 release of the ORBX player, we may explore something along these lines.

At the very least, we'd like to see the ORBX app be launchable from a URI or Android intent that than plays local or remote content with no UI other than what the author specifies in the URI and the packaged content.