Skip to main content
chapdelv
Up and Coming
June 26, 2014

360 S3D camera for the OculusRift

Greetings!

Here is a bit of info on the 360 S3D camera setup that I am using to capture movies for the OculusRift. The setup is composed of 3 cameras each with a fisheye lens. Each camera captures 24 fps (12-bit colors) at 2k x 2k pixel resolution. The cameras are synchronized within 0.1 ms.

The big advantage of this design is that it allows a real-time image stitching method based on epipolar geometry which minimizes misalignments due to parallax. In fact, points along the epipolar lines align perfectly horizontally regardless of their depth. There remains small vertical misalignments vertically but you can remove them for a chosen depth.

You can see a sequence I shot with the camera here:
http://www.iximage.com/street-view-sequence/?lang=en

You can also find more info on the stitching method here:
http://nasv3d.iro.umontreal.ca/chapdelv/pdf/360-s3d-camera.pdf
or the camera itself here:
http://www.iximage.com/?lang=en#technology

Thanks for your feedback.
Cheers.
-Vincent

    36 replies

    Show previous replies
    chapdelvAuthor
    Up and Coming
    June 30, 2014
    "nukemarine" wrote:
    The distortions at eyeline are the stitching distortions that make it look like waves about 15 degrees apart that are running over everything passing by on the street. I don't notice as much with the forward facing movement, but that's only because the parallax between objects is decreased. Nothing can be done about it as you said since it's a three camera system.


    Hi nukemarine,

    I understand your point about the eyes not really pivoting around the nose bridge, but it's unclear to me if this has a significant effect, except maybe for really close objects in the scene...hmm...

    Also, I'm wondering if the fast camera/car motion makes the wavy pattern more visible (BTW, looking at Fig. 9, I'm expecting a peak wave period of about 60 degrees). For a USB 3.0 webcam that you would lie flat on a table, the wavy pattern would probably be less obvious even when using only 3 fisheyes. At least for most people;) You would probably see it as I am guessing that your binocular acuity is excellent.

    Is the wavy pattern as bothering in the Living Room sequence?
    http://www.iximage.com/living-room-sequence/?lang=en

    Cheers.
    -Vincent
    mediavr
    Up and Coming
    June 30, 2014
    "chapdelv" wrote:
    "mediavr" wrote:
    Lensation make a 240 degree M12 lens -- so a Gopro version of your rig is conceivable --


    Unfortunately, the guys at Lensation just replied back that they no longer sell this 240 degree lens... :cry:

    -Vincent


    Damn!

    I inquired about them too recently from a regional dealer and he said too they were no longer listed -- but another lens I inquired about with him turned out to be available when I inquired directly with Lensation. So I thought there might be a chance. The dealer said btw

    Not sure if the BF5M0928ND is still available, it was taken out from Lensation website, but I still have the pricing info.
    SGD 880.00

    Ex works Singapore


    Btw Frank Data had made (by Delft Instruments) a 240 degree fisheye about 20 years ago for a single lens (looking upward) street view type system used in the Netherlands
    http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=1019641

    Btw There is a paper here with test data comparing Fujinon with Omnitech C mount fisheyes
    http://eprints.qut.edu.au/19323/1/Alex_Gurtner_Final_Thesis.pdf
    Nukemarine
    Up and Coming
    July 1, 2014
    "chapdelv" wrote:
    "nukemarine" wrote:
    The distortions at eyeline are the stitching distortions that make it look like waves about 15 degrees apart that are running over everything passing by on the street. I don't notice as much with the forward facing movement, but that's only because the parallax between objects is decreased. Nothing can be done about it as you said since it's a three camera system.


    I understand your point about the eyes not really pivoting around the nose bridge, but it's unclear to me if this has a significant effect, except maybe for really close objects in the scene...hmm...


    I don't know either. That you'd notice it with rendered 3D environments and that's why they included a neckmodel for the camera suggests it's worth looking into.

    Also, I'm wondering if the fast camera/car motion makes the wavy pattern more visible (BTW, looking at Fig. 9, I'm expecting a peak wave period of about 60 degrees). For a USB 3.0 webcam that you would lie flat on a table, the wavy pattern would probably be less obvious even when using only 3 fisheyes. At least for most people;) You would probably see it as I am guessing that your binocular acuity is excellent.

    Is the wavy pattern as bothering in the Living Room sequence?
    http://www.iximage.com/living-room-sequence/?lang=en


    Yeah, I can see the distortions especially on the TV, but since they're not placed on top of a moving scenary like in the roadway video, it's not distracting at all. Aside from not being able to see below 80 degrees, this was a much better video.

    Has there been experimentation with having two feeds (3 top, 3 bottom) and stitching those for a full 360?
    mediavr
    Up and Coming
    July 1, 2014
    Yeah, I can see the distortions especially on the TV, but since they're not placed on top of a moving scenary like in the roadway video, it's not distracting at all. Aside from not being able to see below 80 degrees, this was a much better video.


    The distortions will go away if you up the slices number in Settings in VR Player
    Nukemarine
    Up and Coming
    July 1, 2014
    "mediavr" wrote:
    Yeah, I can see the distortions especially on the TV, but since they're not placed on top of a moving scenary like in the roadway video, it's not distracting at all. Aside from not being able to see below 80 degrees, this was a much better video.


    The distortions will go away if you up the slices number in Settings in VR Player


    Ok, I changed slices from 32 to 64. The distortions didn't go away entirely, but they were much, much more reduced. I don't know if I took a hit on frame rate, but It looks much better. At 96 slices it seemed to go away entirely. Thanks for the suggestion.
    mediavr
    Up and Coming
    July 3, 2014
    Here is a 250 degree M12 lens
    http://www.shtjx.cn/en/index_mcplook_t4_x4_s4_i45.html

    They specialise in ultrawide fisheyes eg. for projection lenses for inside spheres for "Digital Earths" etc

    Here they show a 272 degree lens
    http://www.made-in-china.com/showroom/shitingjixie/product-detailAeimwqFHlvpj/China-Fisheye-Lens.html
    chapdelvAuthor
    Up and Coming
    July 3, 2014
    "nukemarine" wrote:
    At 96 slices it seemed to go away entirely.


    Great! So the distortions were produced by VR player...that explains why I couldn't see them (I'm using my own player).

    "nukemarine" wrote:
    Has there been experimentation with having two feeds (3 top, 3 bottom) and stitching those for a full 360?


    We are doing some experiments with 2 setups to capture the whole sphere. For practical reasons however, I would personnally much rather use only 1 setup (3 cameras) with larger FOV lenses (eg./ 210+ degree FOV) to capture more than a hemisphere (but not quite a full sphere).

    "mediavr" wrote:
    Here is a 250 degree M12 lens

    "mediavr" wrote:
    Here they show a 272 degree lens


    Wow! Thanks for the heads-up. I'll take a closer look at these for sure.

    -Vincent
    mediavr
    Up and Coming
    July 4, 2014
    Here is a listing of their ultrawide CS mount fisheyes
    http://www.chinabagsnet.com/pz5062e2b-cz5c4aa99-monitoring-fish-eye-lens.html
    mediavr
    Up and Coming
    August 6, 2014
    I did a test with 3 Gopros with full circle fisheyes pointing down (Alaud Optical 1.41mm M12). From a 6m pole at an intersection .. and stitched the movie with PTGui and After Effects. With 12cm between each lens pair. Pole shots are a good subject for the technique in the Rift. If a workflow for stitching down and up views could be sorted then you could do the up and down views sequentially (with a 180+ fisheye) without worrying about motion stitching -- with pole shots -- if nothing goes over the visual horizon.

    http://www.reddit.com/r/oculus/comments/2crrfy/360_3d_rift_video_with_three_gopro_cameras_busy/
    chapdelvAuthor
    Up and Coming
    August 7, 2014
    mediavr,

    "mediavr" wrote:
    Here is a listing of their ultrawide CS mount fisheyes
    http://www.chinabagsnet.com/pz5062e2b-cz5c4aa99-monitoring-fish-eye-lens.html


    Unfortunately, most if not all of the lenses in the above link are no longer available. The 272 degree lens you mentioned is also no longer available:(

    Their 250 degree lens IS still available however!
    http://www.shtjx.cn/en/index_mcplook_t4_x4_s4_i45.html

    I ordered one for testing purposes. It's been shipped this morning, I will keep you posted when I get it.

    By the way, I'm downloading your test with 3 GoPro cameras right now. I can't wait to take a look!

    Cheers.
    -Vincent
    Show more replies
    Terms & ConditionsAccessibility statement