Forum Discussion
Resolutions and codec for omnidirectional stereo videos
I'm thinking about rendering some videos with raytracing.
But i'm stuck about choosing resolution and video codec.
-----------------------------------------
About resolution
Oculus Rift / HTC Vive have a 2160x1200 panel. 1080x1200 for each eye.
Consider a FOV of 120°, an ODS image need
360°/120°*1080 = 3240 horizontal pixel and
180°/120°*1200 = 1800 vertical pixel
for each eye, to have a best/correct resolution (not enlarged pixels).
ODS format must have a 4:1 aspect ratio for side-by-side, or 1:1 for top-bottom.
The correct resolution (note: the MINIMUM to avoid that the image is enlarged on the display panel)
for a side-by-side is 7200 x 1800; for a top-bottom is 3600 x 3600.
In case of future generation of HMD with:
the minimum resolution for a side-by-side will be 12960 x 3240; for a top-bottom 6480 x 6480.
In case of future generation of HMD with
the minimum resolution for a side-by-side will be 24576 x 6144; for a top-bottom 12288 x 12288.
To be more precise, for calculate the perfect resolution it's necessary to know to exact FOV of the HMD.
Other people that i found about this topic say that:
-----------------------------------------
About Video Codec
So, i rendered (using POV-Ray) an ODS 3600 x 3600 top-bottom test video of ten seconds, at 75 FPS (note: not 90 FPS...), encoded with x265 (ffmpeg).
My test (on a Intel Core i7-6700 CPU @3.40Ghz 8 CPU - GeForce GTX 980 Ti - Windows 10):
-----------------------------------------
Questions
But i'm stuck about choosing resolution and video codec.
-----------------------------------------
About resolution
Oculus Rift / HTC Vive have a 2160x1200 panel. 1080x1200 for each eye.
Consider a FOV of 120°, an ODS image need
360°/120°*1080 = 3240 horizontal pixel and
180°/120°*1200 = 1800 vertical pixel
for each eye, to have a best/correct resolution (not enlarged pixels).
ODS format must have a 4:1 aspect ratio for side-by-side, or 1:1 for top-bottom.
The correct resolution (note: the MINIMUM to avoid that the image is enlarged on the display panel)
for a side-by-side is 7200 x 1800; for a top-bottom is 3600 x 3600.
In case of future generation of HMD with:
- 4K UHDTV / 2180p for both eye (3840 x 2160)
or - 4K DCI for both eye (4096 x 2160)
or - Current Oculus Rift / HTC Vive panel for each eye (2160 x 1200 x eye)
or - 1080p for each eye (1920 x 1080 x eye)
the minimum resolution for a side-by-side will be 12960 x 3240; for a top-bottom 6480 x 6480.
In case of future generation of HMD with
- UHDTV / 2180p for each eye (3840 x 2160 x eye)
or - 4K DCI for each eye (4096 x 2160 x eye)
the minimum resolution for a side-by-side will be 24576 x 6144; for a top-bottom 12288 x 12288.
To be more precise, for calculate the perfect resolution it's necessary to know to exact FOV of the HMD.
Other people that i found about this topic say that:
- DK2 has a ~94°x106° FOV, the correct resolution is 4962 x 4962 in top/bottom.
- If CV1 has the same FOV of DK2, 5502 x 5502 in top/bottom.
- HTC Vive, 5234 x 5234 in top/bottom.
-----------------------------------------
About Video Codec
- H264:
The max resolution for the x264 highest level (5.2) is 4096×2304.
All resolutions above are outside this limit. - HEVC/H265:
The max resolution is 8192 x 4320. It fit only the worst resolution of current generation of HMD.
So, i rendered (using POV-Ray) an ODS 3600 x 3600 top-bottom test video of ten seconds, at 75 FPS (note: not 90 FPS...), encoded with x265 (ffmpeg).
My test (on a Intel Core i7-6700 CPU @3.40Ghz 8 CPU - GeForce GTX 980 Ti - Windows 10):
- The latest VLC crash at play.
- The latest MPC-HC fill the CPU, and can't playback smoothly.
I discover that GTX 960 and above have an H265 hardware accelerated decoder (in DXVA mode), but in MPC-HC it's disabled by default.
I enable it, some less CPU load, but anyway unplayable. - On Rift, Virtual Desktop refuse to open the file.
- On Rift, MaxVR playback the file, but not smoothly.
-----------------------------------------
Questions
- "Oculus Video" that will shipped with CV1 (or SDK 1.0) will have h265 hardware accelerated support?
- Anyone know a Video Player optimized for high resolution ODS?
- Exists a a Video Player that can open 2 video (both h264 hardware accellerated) simultaneous and display on two hemispheres?
5 Replies
- Have you considered the CUDA video decoder that can decode AVCHD directly to an OpenGL or DirectX surface: http://docs.nvidia.com/cuda/video-decod ... z432yPGjV0
360°/120°*1080 = 3240 horizontal pixel and
180°/120°*1200 = 1800 vertical pixel
for each eye, to have a best/correct resolution (not enlarged pixels).
I think you are not taking lens distortion into account.
The GearVR for example has a 90° FOV on a 2560x1440 panel, but the center pixel covers 0.06° after distortion. You need this value of pixel coverage or "pixel density" or "pixel per display pixel" or "eye buffer scaling" as it's variously called.
For the GearVR, 0.06° pixels means we need 360/0.06 = 6000 pixels to cover one monoscopic turn. This is different from the value you would get with your formula (5120) because you'd assume that pixels cover ~0.07°. What it is saying is that to create an image that takes full advantage of the display panel, you need not render at the nominal 1280px width but at 1500px.
Have you considered the CUDA video decoder that can decode AVCHD directly to an OpenGL or DirectX surface:
Here say AVCHD 2.0 support H.264 High Profile (HP) up to H.264 Level 4.2... a very low resolutions for ODS.
I'm following this viewer, but it don't support video yet.
Do you know other video player that playback video with CUDA?
I recently write and tested the support of ODS in POV-Ray.
I'm hobbyist, i not aim to develop a video player.
Under my possibility, i'm thinking about create a scene, maybe builded with the feedback of this community, that can be a test/sample video for testing HMD display, video player performance, codecs etc.
Something simple but effective. Think about a checked plane (for checking projection), floating spheres (for checking 3D), dark & light area (for checking display, gamma etc.), axis (for checking orientation) etc.
Rendered in a lots of resolutions and different FPS, and shared as project in this forum with sources and all instruction about re-render in other format.
For the GearVR, 0.06° pixels means we need 360/0.06 = 6000 pixels to cover one monoscopic turn.
Worse still about choosing a codec, a 6000 x 6000 will be outside any profile of HEVC/H265 (and of course H264).
Do you know the center pixel cover X° of CV1 or Vive?- I'm interested to know if there is a player that can play back 12k 360 (or higher resolution) video.
I can produce the video by stitching with autopano video but i don't see any way to play it - For video decoding able to handle any resolution at 90 fps (CUDA based solution) see www.depthq.com and in software section look for DepthQ VR Player. The code pass through PCI-E compesed data and the decoding is 100% executed in GPU.
