Skip to main content
Up and Coming
June 26, 2014

360 S3D camera for the OculusRift

Greetings!

Here is a bit of info on the 360 S3D camera setup that I am using to capture movies for the OculusRift. The setup is composed of 3 cameras each with a fisheye lens. Each camera captures 24 fps (12-bit colors) at 2k x 2k pixel resolution. The cameras are synchronized within 0.1 ms.

The big advantage of this design is that it allows a real-time image stitching method based on epipolar geometry which minimizes misalignments due to parallax. In fact, points along the epipolar lines align perfectly horizontally regardless of their depth. There remains small vertical misalignments vertically but you can remove them for a chosen depth.

You can see a sequence I shot with the camera here:
http://www.iximage.com/street-view-sequence/?lang=en

You can also find more info on the stitching method here:
http://nasv3d.iro.umontreal.ca/chapdelv/pdf/360-s3d-camera.pdf
or the camera itself here:
http://www.iximage.com/?lang=en#technology

Thanks for your feedback.
Cheers.
-Vincent

    36 replies

    Up and Coming
    August 7, 2014
    It will be interesting to hear about the 250 degree fisheye! I asked about it too and the lack of an IR-cut filter (and the $400 price tag) was enough for me to think twice about getting one to test -- I think it would be very good though
    chapdelvAuthor
    Up and Coming
    August 9, 2014
    mediavr,
    Your test shot with GoPro cameras is very nice! I like your idea of putting the camera rig upside down and very high, a bit like an S3D security camera...Did you use any device to synchronize the cameras together?

    If you don't mind giving me a link to download the 3 captured videos, I can try to stitch them together using my software and see if the result compares to your PtGui solution. Let me know.
    Cheers.
    -Vincent
    Up and Coming
    August 10, 2014
    I like your idea of putting the camera rig upside down and very high, a bit like an S3D security camera...Did you use any device to synchronize the cameras together?

    If you don't mind giving me a link to download the 3 captured videos, I can try to stitch them together using my software and see if the result compares to your PtGui solution. Let me know
    .

    Vincent
    I did not use any sync device but I shot at 48fps so I would have a better chance of getting ok sync than at lower fps. I used the Gopro Wifi remote to start and stop them but that gives you a few frames error generally and I improve on that by looking at sound spikes in the sound tracks and dragging the videos into alignment on the timeline. In fact, going on those the sync was ok for two cameras (maybe 2msec missync) and not ok really for the other one (about 10msec missync).

    With my Gopro Dual Hero System 3d sync backs I could have got excellent sync with two cameras. Folks are beginning to look at how to sync more than two Gopros with hacks of the Dual Hero System
    http://goprouser.freeforums.org/gopro-dual-hero-cable-reverse-engineer-t19093.html
    but I am not very optimistic.

    The Blackmagic Personal Cinema Camera is getting attention from 3d film makers lately -- sync wise -- I think that might be a better chance. These would be nice cameras for fisheye video (raw video, high dynamic range, cheap, compact)
    http://personal-view.com/talks/discussion/10902/bmd-pocket-cinema-camera-stereoscopic-collaboration.

    Here is a zip of the three movies (I send the roughly cropped and aligned sequences rather than the much longer raw videos)
    https://www.dropbox.com/s/pflv2ypapsmxueh/hemi.zip

    My stitching in my final output is a bit rough near the nadir -- I was very quick and approximate with my blending mask making.

    Btw
    I looked at your original samples in Whirligig and they work well in that in the DK2 -- Whirligig has a "Stereo Dome" mode. I would like Whirligig better if you could change the viewing fov like in VR Player .. but the scrolling does seem smoother.
    chapdelvAuthor
    Up and Coming
    August 14, 2014
    mediavr,
    I stitched your sequence last night. The result is probably a little better near the nadir, but this is understandable as my software was specifically designed for this camera setup. Or maybe PtGui could achieve the same result with more work, as you say. Anyways, here you go:
    http://nasv3d.iro.umontreal.ca/chapdelv/mediavr/qvboverunder-2.mp4

    I'm still amazed at how little your 6m pole moved! Did you just hold it tightly against yourself?

    Yes, I know about Blackmagic cameras. They seem to be great little cameras. When I designed my camera setup, my final choice was between these and the Mako G-419 cameras. I finally chose the latter so I could easily do some live capture+stitching+display with a standard computer and an OculusRift.

    Cheers.
    -V.
    Up and Coming
    August 15, 2014
    http://nasv3d.iro.umontreal.ca/chapdelv/mediavr/qvboverunder-2.mp4

    That looks very nice stitching-wise! I will have to be more careful with my masks.

    I'm still amazed at how little your 6m pole moved! Did you just hold it tightly against yourself?


    I use poles a lot for regular panoramas -- so I have developed gear for it -- eg. this base I used for this shot

    ... a hacked old tripod with a giant leveling ball and a clamping arrangement
    http://www.mediavr.com/levellingbase.jpg
    now I use a Nodal Ninja pole with that base
    http://shop.nodalninja.com/carbon-fiber-poles-all-1/

    re camera configurations -- I have worked out a way of having a triangle of Gopros with a lens separation of 50mm with two of the cameras linked by the Dual Hero System for sync (and with all three cameras oriented correctly for identical rolling shutter ) -- two cameras synced means perfect sync at least for 120 degrees, 60 degrees front and back, say.

    Also there is a possible configuration, with the lenses touching, for minimal lens separation with the Gopros/Alaud Optical 1.41mm, viz. 30mm but with varying rolling shutter.

    I have made rigs for these layouts and will test them soon.
    Up and Coming
    August 17, 2014
    How much VFoV would 3 standard GoGpo cameras and lenses cover? Would it be enough to achieve 3D in the zenith? What about 4 cameras?
    If this was done to cover the zenith, then 4 pairs in landscape could cover the horizon.
    Another 3 or 4 to cover the nadir.
    This would be an extension of EleVR setup. http://elevr.com/tutorial-3d-spherical-camera-head/
    Up and Coming
    August 20, 2014
    I made a post about a 50mm interaxial Gopro rig, with some pictures and a video download (side by side, 180 fisheye), on the Oculus subreddit

    http://www.reddit.com/r/oculus/comments/2e1u6e/a_compact_gopro_rig_for_hemispheric_stereoscopic/
    chapdelvAuthor
    Up and Coming
    March 5, 2015
    Hi everyone,
    I just wanted to share a new version of the rig. While the previous version aimed realtime capture/stitching/viewing in the Rift, this one was made to be more portable. Each camera is a modified GoPro Hero 4 Black (which can capture at 4K resolution). Oh, and each lens has a 250 degree FOV!

    I still have to perform tests w.r.t. how well the cameras are synchronized together. I am using the GoPro remote but I am not sure if this is any improvement to the worst case of, say, 16ms when shooting at 30fps. Is someone aware of any serious tests on this issue?

    Also, would someone know of a custom GoPro firmware that would allow a square 2880x2880 capture at 24fps or 30 fps (or even more, if possible, to limit the desync between the cameras)?

    I'll keep you guys posted when I have a few video sequences to show.

    Cheers.
    -Vincent
    Participant
    March 6, 2015
    Im curious about your lenses, where did you acquire them, and for how much, and finally hows the performance of the lens?
    Lee Bledsoe Immersion:RealLife http://www.imreallife.com
    Up and Coming
    March 11, 2015
    Yeah, my lenses are only 170-degree (for my current image sensor chip dimensions). :cry: Where do I get these "more better" lenses?
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.