Skip to main content
j1vvy
Up and Coming
January 24, 2014

Experimenting in Panoramic Stereoscopic 3D video

I have a 360° Panoramic stereoscopic 3D multicamera video rig using 14 cameras.
http://photocreations.ca/3D/mobius_camera_rig.html

I have added a new S3D panorama of the sun setting.
http://photocreations.ca/3D/index.html
Unfortunately two of the cameras did not record. The faulty cameras were the left and right view of the exact spot where the sun was going down. The -20°C cold weather meant the cameras did not record as long as I hoped. record time were between 20:48 and 53:50, averaging 37:30 minutes. I compressed the usable 20 minutes footage into a 1 minute timelapse.

I was also testing the quality of locking the white balance of all the cameras; in this case to cloudy. I corrected for the changes the cameras make because of autoexposure but even with whitebalance locked, I should have also corrected for differences in whitebalance.

I have got much better at the aligning and stitching of the videos.

My new interest in 3D panoramic video led me to go back and render in higher resolution my first interactive panoramic video I have published. I shot it in April 2006 before I had any way of showing it.

It is a poker game with the camera positioned over the center of the table. I used my full frame fisheye on my Nikon D70s. I shot my VR video by pointing my camera in different directions for each sequence. I tethered the camera to my computer capturing a jpg image as frequently as I could, every couple seconds.

I published a smaller interactive version in 2008. Over the years I have been told that it has inspired others in the panoramic photography community to create interactive panoramic video. Every time I watch it I want to make it again with a proper video camera and better actors.

To watch it, get the full story, and see how it was made see my site.

http://photocreations.ca/mff2008_wpt2006/index.html

    29 replies

    Show previous replies
    j1vvyAuthor
    Up and Coming
    May 3, 2014
    The nodal point offset is an issue with multi-sensor camera rigs.
    There will be parallax error joining the images for any distance except one.
    Similar to the circles of confusion when describing focus for a camera and how it is effected by F-stop,
    When the sensors are at the same nodal point the image is aligned from the camera to infinity. When the sensors are not at the same nodal point it is possible to choose when the images will align and the closer together the sensors are the more depth is in alignment.
    So the goal of a multi-sensor camera rig is to get the sensors as close together as possible to minimize the parallax error.

    With a 3D rig it is necessary to have the cameras separated. Ideally equal to eyes viewing, but I aim for a 65mm average.

    Combining these two was a challenge. Having the overlapping cameras was a big breakthrough and allowed both minimum distance between sensors for each pano while maintaining a larger distance for separation for left and right eye.

    Having stereoscopic 3D in the zenith and nadir would only work when viewed from one direction. So my plan was to have none. The 3D would slowly fade to 2D.
    j1vvyAuthor
    Up and Coming
    June 29, 2014
    Added videos of Bellagio Fountains while in Las Vegas for the IVRPA conference. There are 3D and mono versions.
    I think you will find the 3D version much more immersive. Like to know what you think.

    http://photocreations.ca/3D/index.html
    HiroProtag0nist
    HiroProtag0nist
    Honored Contributor
    August 9, 2014
    I've just found the wonder of these videos and the Whirligig player, but have noticed only the lower resolution videos will play (New Years Fireworks and Bellagio Fountain 3d).

    Just tried Sunset New Brunswick (360MB) and nothing.

    What is preventing the higher resolutions - the player, the Rift or my PC - anybody else ok getting these to play?
    Big PC, all the headsets, now using Quest 3
    chuckchae
    Participant
    August 23, 2014
    Very nice and impressive readings.. Thank you .
    chuckchae
    Participant
    August 23, 2014
    "j1vvy" wrote:
    The nodal point offset is an issue with multi-sensor camera rigs.
    There will be parallax error joining the images for any distance except one.
    Similar to the circles of confusion when describing focus for a camera and how it is effected by F-stop,
    When the sensors are at the same nodal point the image is aligned from the camera to infinity. When the sensors are not at the same nodal point it is possible to choose when the images will align and the closer together the sensors are the more depth is in alignment.
    So the goal of a multi-sensor camera rig is to get the sensors as close together as possible to minimize the parallax error.

    With a 3D rig it is necessary to have the cameras separated. Ideally equal to eyes viewing, but I aim for a 65mm average.

    Combining these two was a challenge. Having the overlapping cameras was a big breakthrough and allowed both minimum distance between sensors for each pano while maintaining a larger distance for separation for left and right eye.

    Having stereoscopic 3D in the zenith and nadir would only work when viewed from one direction. So my plan was to have none. The 3D would slowly fade to 2D.


    Impressive.
    j1vvyAuthor
    Up and Coming
    August 25, 2014
    Added a video of watching firework from the middle of a crowd of people.

    Turning around and looking at the people the sense of 3D is very obvious and increases the immersive experience.

    But I find the sense of 3D in the fireworks to be barley visible.

    The short light flashes at the end of the fireworks shows how important it is to have perfectly synced cameras. I was shooting at 30fps and then synced in post production to the nearest frame. The flashes at the end were so short that with some cameras they happened between frames, and were not captured, but show up in others cameras. It is very disorienting to watch, but only last a few seconds at the very end.

    http://photocreations.ca/3D/index.html

    Thanks for the kind words.
    I'll update the page with Whirligig player instructions to watch my videos in a bit.
    Amnio
    Participant
    September 24, 2014
    This is the best 3D 360 video I've seen so far. Almost felt like being there. Incredible stuff, really amazed.
    ErikCaretta
    Participant
    January 12, 2015
    Hi j1vvy,

    Your content looks pretty cool. I agree it seems to be the best 3D 360 out there. I read this thread and your website, we came up with pretty much the same considerations about rig configuration (I'm also interested in stereo panorama shooting).
    But what I'm most concerned about, at the moment, is the lack of an easy and reliable way of watching 360 stereo videos, at a resolution of at least 3K and with good results.

    I tried several players:

    - VRplayer: can't have the positional tracking to work on DK2, and beside that the render quality doesn't look great to me
    - Kolor Eyes: it works, and it's simple, but looks like it doesn't use hardware acceleration on windows, so it's pretty unusable at the moment
    - Whirligig: I couldn't even make it work with anything different from the included couple of jpegs.
    - LiveViewRift: it works, but as Kolor Eyes it doesn't seem to use hardware acceleration

    Since a significant effort is needed to build a rig and a professional workflow to have this kind of videos done the right way, I'm asking myself if it's worth right now or if maybe we're a little bit early.

    I mean, I'm not an expert on the field but I've been working in CG and video production for years now so I'm not a 100% newbie: if I'm encountering similar difficulties, how many people in the world can I expect to be able to correctly and easily watch my video, and above all, enjoy it?

    What is your opinion on this?
    Can you run 4k stereo panoramas without lags?
    j1vvyAuthor
    Up and Coming
    January 12, 2015
    It took a few things to get VRPlayer to work on my PC but since then tracking has been working without issue.

    Whirligig and LiveViewRift both work for me.

    On my 6 year old PC with new graphics card, I am limited to about 2.7k stereo. In mono I can not play 4k.

    I have not tried virtual desktop, which I believe supports hardware rendering for video.

    On my 2 year old Android phone with VRPlayer I can not play 2k stereo but can play 1k stereo. I have not experimented too much with different sizes. GearVR can handle even more and smoother.
    Anonymous
    January 14, 2015
    Hi....I've read your web and it really impress me....I've researched the stereo panorama several days. and I know that in the stereo panorama, the zenith and nadir can not be resolved because when you yaw 180° it's a inverse parallex.
    but I want to know the mean of "The 3D would slowly fade to 2D" , do you mean at zenith and nadir the parallex is zero? and if your camera is 120°HFOV(as your web describe) , how can you get the another 60° at zenith and nadir?
    thanks very much!

    "j1vvy" wrote:
    The nodal point offset is an issue with multi-sensor camera rigs.
    There will be parallax error joining the images for any distance except one.
    Similar to the circles of confusion when describing focus for a camera and how it is effected by F-stop,
    When the sensors are at the same nodal point the image is aligned from the camera to infinity. When the sensors are not at the same nodal point it is possible to choose when the images will align and the closer together the sensors are the more depth is in alignment.
    So the goal of a multi-sensor camera rig is to get the sensors as close together as possible to minimize the parallax error.

    With a 3D rig it is necessary to have the cameras separated. Ideally equal to eyes viewing, but I aim for a 65mm average.

    Combining these two was a challenge. Having the overlapping cameras was a big breakthrough and allowed both minimum distance between sensors for each pano while maintaining a larger distance for separation for left and right eye.

    Having stereoscopic 3D in the zenith and nadir would only work when viewed from one direction. So my plan was to have none. The 3D would slowly fade to 2D.
    Show more replies
    Terms & ConditionsAccessibility statement