Minimum FPS for 360 3D VR video

It's only the app itself that needs to run at 90fps (or 75fps on DK2). This is mostly so the head-tracking is responsive (which the mind is very cognizant of). Having the video itself running at a lower rate would probably still be acceptable. Of course, if you CAN record at 90Hz that's even better, but if that's not possible it's not the end of the world.

"cybereality" wrote:
It's only the app itself that needs to run at 90fps (or 75fps on DK2). This is mostly so the head-tracking is responsive (which the mind is very cognizant of). Having the video itself running at a lower rate would probably still be acceptable. Of course, if you CAN record at 90Hz that's even better, but if that's not possible it's not the end of the world.

Yeah, a slow framerate video is like a fast slideshow. Perfectly normal to view (on a virtual screen) as long as head tracking is fast. Slowly changing objects in the environment may look unusual, but not dizzy-making, as long as the environment (static or dynamic) appears where your mind expects to see it while moving your head.

Thanks for the info,

I was thinking to use the upcoming Black Magic Micro Studio Cameras, so i'm glad to know I could use them.

"John00100" wrote:
Thanks for the info,

I was thinking to use the upcoming Black Magic Micro Studio Cameras, so i'm glad to know I could use them.

The actual video content only needs to change frames often enough to be perceived as continuous motion, typically 15 FPS in normal lighting, or as little as about 7 FPS in low lighting. Though with fast moving objects you obviously need either high update rate or motion blur, to help your brain perceive objective continuity (perceived as same object in consecutive frames). The display, however, must track your head motion with high update rate, perhaps using async timewarp or similar techniques (such as "PTZ tweening"), which is perceptually related to motion picture projectors displaying each frame multiple times to prevent perception of annoying flicker. Except for VR, each of those frame repetitions must be moved in visual space to where your brain expects to see it to prevent motion sickness.

Ah thanks, I get it now.

"geekmaster" wrote:
... The display, however, must track your head motion with high update rate, perhaps using async timewarp or similar techniques (such as "PTZ tweening"), which is perceptually related to motion picture projectors displaying each frame multiple times to prevent perception of annoying flicker. Except for VR, each of those frame repetitions must be moved in visual space to where your brain expects to see it to prevent motion sickness.

Even better than just head-tracked warping of repeated frames would be to modify those repeated frames to morph over time from one input frame to the next (i.e. "Keyframe Tweening"). You could create such in-between frames using video codec motion vectors to do smooth interpolated framerate upconversion, perhaps with video encoder hardware assistance. These ideas were discussed in my "PTZ Tweening" thread at MTBS3D:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=16543

If our video source comes from digital video cameras, just use the embedded motion vector data in the compressed video stream to do fast and simple frame interpolation, adjusted to match head position for each screen update (perhaps even predicting scan line pixel time for each "rolling shutter" line update).

For stereo 360 video you do want the cameras to capture each set of frames at the same time. If not using cameras that can be synced, then using a higher frame rate allows to capture them closer together.

Capturing at 90fps and only process every other frame for smoother playback might be a better option.

With extreme high fps GoPro will show visible rolling shutter which is worse, especially if the camera is moving.

"j1vvy" wrote:
For stereo 360 video you do want the cameras to capture each set of frames at the same time. If not using cameras that can be synced, then using a higher frame rate allows to capture them closer together.

Capturing at 90fps and only process every other frame for smoother playback might be a better option.

With extreme high fps GoPro will show visible rolling shutter which is worse, especially if the camera is moving.

There are some excellent algorithms that use motion vector interpolation to tween the missing frames. You just need to use differential time sync to interpolate motion vectors while building a synchronized stereoscopic views. I have followed this research for more than a decade, and there have been huge advances in recent years. You can interpolate both time and space interchangeably or in combination, also allowing dynamic adjustment if IPD in post processing, in addition to synchronizing (genlocking) multiple cameras in post processing.

Sure, if you have the budget, camera genlocking saves the need for such post processing, but said post processing allows us to do amazing things with low-budget or existing stock footage that was not genlocked.

And with modern GPUs and/or custom ASIC video codec hardware, it may be useful to use such hardware to acquire the motion vectors for us...