cancel
Showing results for 
Search instead for 
Did you mean: 

Discussion points concerning Live Action VR Cinema. Is it really here and ready?

Gerald1W
Honored Guest
Hey guys,

Posted this elsewhere but then realised I'd probably done it in the wrong section, so re-posting because I think this is the correct area.

First up primarily I'm a cinematographer (not a developer or VR wiz) and I've got a feature film script on my desk which I believe lends itself to the immersive/VR cinema idea perfectly. It has 3 defined acts, each act in 1 single small location (rooms within a house). 

In my research though I'm coming up against a fair few issues I guess which are concerning and this seems to be the best hive of activity for all things VR related so thought I'd post and see what answers I can get.

1.
I haven't come across any examples of live action narrative story telling using VR that I find engaging.

I've looked at some of the material from Jaunt but their use of the 360 degree camera seems to mostly be as a gimmick.

Can someone point me to an example of a production which has used high end equipment (red Epics perhaps?) which has followed through and has maintained its quality to distribution.

2.
Speaking of distribution, I'm concerned that if making a feature film in any VR format it seems to be locked in to 1 distribution hardware/platform.

Can someone (if its possible?) talk me quickly through the process of taking footage from a multicam Red rig like this:

http://www.cinematographydb.com/wp-content/uploads/2014/11/Volvo-Header02.jpg

And then once it is stitched together... is it possible to output it so different platforms can view it (i.e. rift/vive/PSVR/cardboard) ? OR is it that you pick your platform first and from pre production to post production you are locked in to 1 avenue.

3.
Speaking of Google Cardboard... and thinking about codecs... it would appear that h264 can not handle the data required for 360 video. Even with a 90 degree viewing angle... a 360video requires 4x the bitrate of a regular HD video. Now because compression rates are already set on youtube, it means the compression for each view is 4x greater so most of the videos you can watch on youtube seem extremely blocky/low res. The playback is truly awful. Does anyone have any insight in to codec support for the different VR systems and if anyone will have native h265 support which COULD potentially support the higher bit rates required for 360 video. 

Doing a rough calculation my 70minute feature film, if encoded for h265 and played on screens of similar resolution to a Rift, would require 11gb of video. Anyone have any discussion/contributions as far  as this as being delivered viably?

4.
It would seem the majority of 360 degree videos provide the viewer with an angle of view of between 90 and 110 degrees. For anyone familiar with cinematography/lenses/photography this equates to a 9mm lens on an APS-c (super 35mm motion picture format) or a 14mm lens on a full frame camera (for those who have shot 35mm film stills). 

These are EXTREMELY wide lenses and don't offer very flattering images and also offer extremely exaggerated foregrounds and backgrounds.

Does anyone know if people have shot with tighter lenses and then also offered tighter viewing on the distribution end? Ideally I'd be aiming for around 50-55 degrees of field of view which is slightly closer to the humans field of view than the extremely distorted views we are used to in VR.

5.
Audio! Is 50% of a film. And I'm very worried that anyone listening using headphones will become disorientated when they are turned 180 degrees from the action and hear sounds in the wrong ear.

This post from Story Studio covers these concerns but I'm wondering what would then happen when you try and output the video for a different VR headset. I guess these are standards that aren't accepted and every delivery would require a new soundmix?

Final thoughts

If anyone has read/thought about/might have answers to any of these questions I'd LOVE to hear any ramblings, thoughts or comments.

I guess in my ideal world, it is possible to shoot in a tighter field of view, with high end equipment... Then stitch everything together and output to multiple different delivery options (different VR headsets). The playback codecs will catch up with the intense amount of data required to output a high quality image (something I haven't seen yet in any example) while the audio is also aware of the direction of the viewer.

I believe these points are major hurdles blocking the progression of VR and Cinema combining.
7 REPLIES 7

RobHermans
Superstar
@Gerald1W :
It's a shame that you've made the effort to write such an extensive post and received no replies yet. Unfortunatley for you, this area of the forums is mostly frequented by end users (I'm in that group myself!), and many of your questions/points go right over my head... but surely there are some guys out there who can respond?

nosys70
Expert Protege

1.
You could check what has been done for some entertainement park with 360 movie.

Most of them are short "gimmick" movie, but there is still some know-how inside.
Since these are relatively old movie, they used bulky rigs with mirrors but quality was extremely high.
2.
360 movie (stop speaking about VR, since these kind of movies have nothing related to VR)

they are just full sphere 360 movie and if lucky 3D 360.

There is no limitation to play such movies, you can play them from a tablet/smartphone, from a VR helmet etc...

The only limitation will be the quality of the image on the device, and unfortunately , in this era of 4k TV screens, the picture quality of devices is VERY low.

So shooting with a bunch of red or a bunch of GoPro would probably make no difference in a google cardboard or even in an Oculus.


3.
Google cardboard is just ... cardboard (and oculus is just plastic) , so codec has nothing to do with the device. Most of smartphones have very limited power and can hardly play better than full HD at high framerate. and the codec you will use will depends from the ability of the device to use it. Even a Samsung S7 or an IPhone 6S will probably accept different codecs.

Currently, the state of the art is H265, but even there, there is many variation within a codec (framerate, bandwith, compression ratio, level) that can make it unplayable on a specific device. You chances are greater with a PC, but audience should be bigger with smartphones. (everybody has a smartphone, while people with a VR device and a good pc are still exception).

There are hundred of smartphone models you can slip into a google cardboard, so you best guess is to deliver a movie in the minimum requirement playable form (full HD , H264, 30fps)

4.

A 360 deg video as the name implies, is 360, so there is no point to speak about viewing angle , this is the fact from the viewing device (mostly between 90 and 120).

What is important is the projection must match a real view, too wide or too close  and it looks strange..

that is the problem of 360 video, is you cannot cheat with zoom, framing, focus  like in cinema.

The balance must be found between few cameras with wide angle (and you lack definition in the background) or lots of camera with narrower lenses.

Whatever you do, you will to stack enough cameras and lenses to cover 360deg of fov, with overlap between cameras for stiching.

If you choose to shoot with 50mm lense (about 30deg field of view, you will need around 20 cameras to cover 360 deg (horizontally) and more to get full sphere.

So you understand why people are looking for wide angle lenses.

5.
Since audio is possible in 5.1 channel, 360 audio should be easy to implement.

unfortunately headphones able to render spation sound are like VR headset, still exception.

Conlcusion.

There is no technical limitation to shoot 360 movie with sound, out of the fact it will probably an adventure like first 3D movies were.

Playing the movie in good condition is yet almost impossible out of a special equipped room, but you could easily broadcast it on youtube or internet, provided you forget anything related to "quality".

You can easily do a test for cheap by purchasing a 400$ ricoh theta and playing a few shots right into an oculus or a vive.

Greyman
Superstar
Hmmm, I'm not sure that VR films wouldn't be possible or a good idea.  Imagine being able to take the role/perspective of a choice of characters in each scene.  The victim or the killer in a VR remake of the Psycho shower scene for example.  Maybe not a good example, but an interesting possibility nonetheless.

nosys70
Expert Protege

there is a problem with vocabulary here.

VR is immersive virtual reality. If you remove immersive, 3D and 360 vision, you just have a plain good old movie on a screen

Nobody said that 3D movies are VR, and 180 or 360 movies exists way before VR, and nobody said there are VR.

So you can take advantage of the VR hype, allowing you to display 3D, 360 (or both) in an immersive way thanks to these new VR devices, to display movies in a new way (gimmick ? as 3D was often named ?).

The fact you have a scenario that support the new feature (and here I do not speak about 3D, or 360 video that we already know for years, if you have been in an entertainement park the last 30 years).

The real question is are you able to change the paradigm of a movie, where people are safely seated as witness of a story (and so , can look at the story with this particular point of view) to another paradigm where the spectator becomes part of the story (because he is now in the action) and all is made to make him believe his integrity/opinion could be put in the balance. (for example just like the fact he can now choose where to look for).

take a simple scene where there is some risk in the story (explosion, free fall, any kind of attack).

In a regular movie theater, you are comfortably seated looking at the screen, and curious of what could happen to the hero.

you do not fear flames, explosion, or zombies, since you know you are  "here" and the story plays "there". At best you get some thrill.

Now imagine that the director makes everthing he can (add 3D, 360 picture, surround sound, even vibrating seats, plug a VR headset on your head) to make sure you feel as you are "there" and so, the incoming buffalo herd will just smash you at the same time as the hero.

The problem is you know the hero got the trick to save his life, but you are not the hero...

you are just you , teleported in a scene where suddenly you could not care about the story, but about what happens to you.

These examples are a bit exaggerated, but for example, when a scene take place in the street, you do not care about walking people and running car. Just about the dialogue between starky and Hutch.

With Vr headset, since you are "there" you would probably mind about the the traffic (cars on the street and people on the sidewalk as you would do in reality.

So as director, you would be constantly teared between the need to put the spectator into the scene (using all these nice tools from VR) and keeping the "distance"  to not hurt the spectator's mind .


A really simple analogy would be to imagine the difference between looking a guy driving a car like a mad and be seated beside him in the car.

The view is the same , the feeling is totally different.

I can imagine, there are some way to take advantage of this kind of situation, but else, this would limit "VR" movies to very boring movies where almost nothing happens except dialogues between characters.



Greyman
Superstar
I would just like to be in Neo's shoes when he does the jump programme or in Trinity's in the hallway battle scene 😉

jcasman
Explorer
This thread is quite old but since I've been involved with spatial audio recording using the RICOH THETA (360 degree camera) recently I thought I'd add just a little information here. This is in response to topic #5 from the OP. The new RICOH THETA V boasts 4 microphones internal to the camera, and spatial audio is a key feature. When used with earphones or with a VR headset, the directionality of audio significantly enhances the overall experience. This interview with Michael Wohl, award-winning videography with extensive experience in 360, adds tons of details for end users (not exclusive to RICOH THETA): Effectively Using Spatial Audio - Interview with Michael Wohl


jcasman
Explorer
This thread is quite old but since I've been involved with spatial audio recording using the RICOH THETA (360 degree camera) recently I thought I'd add just a little information here. This is in response to topic #5 from the OP. The new RICOH THETA V boasts 4 microphones internal to the camera, and spatial audio is a key feature. When used with earphones or with a VR headset, the directionality of audio significantly enhances the overall experience. This interview with Michael Wohl, award-winning videography with extensive experience in 360, adds tons of details for end users (not exclusive to RICOH THETA): Effectively Using Spatial Audio - Interview with Michael Wohl