Cutscene Alternatives?

This is an interesting topic and is part of understanding the intersection of VR and cinematography as a whole. Part of cinematography is the manipulation of the frame, the field of view, the depth of field, the lighting, etc in order to convey an emotion or idea.

However, the language of cinema doesn't necessarily translate 1:1 to VR. Sudden transition of the viewpoint in film, or 'cuts' are so common in that it's considered notable when you go for long periods without them. But as you say, they can be extremely disconcerting when done in VR, where you've increased the level of immersion specifically to make the viewer feel more like they're present in the scene than simply a viewer. This means that common things like the 'Angle / Reverse Angle' sequences used to alternate between closeups of two people speaking as they converse is something that doesn't really work in VR.

The problem is, how do you convey certain non-verbal ideas without saying them. Cinema itself had this very problem, because it was a medium that grew out of the effect of disruptive technology on a prior storytelling medium, the stage, which in turn had to deal with the same problems.

Consider two people trying to communicate privately without other parties nearby hearing. For instance, this exchange from Romeo and Juliet:

SAMPSON: [Aside to GREGORY] Is the law of our side, if I say ay?
GREGORY: No.

It includes the stage direction Aside to GREGORY indicating to the actors that this is intended to be a private question asked of one person to another and not heard by the other people in the scene. In stagecraft, this can be done with exaggerated body language, where you rely on an actors skill at conveying emotion and ideas with his body as a whole, because the audience doesn't necessarily have a good view of the actor's faces. In cinema, the point of view is no longer fixed, so you can convey the idea by actually cutting from a wide view of the scene to a shot that only includes these characters, close enough to convey a sense of intimacy. This is then close enough to allow the actor to whisper and yet the viewer from this new perspective can plausibly hear the exchange.

Stagecraft solves the problem one way, cinema another, both suited to the constraints of the medium. Before new techniques were developed and refined, most cinematography was essentially nothing more than filmed stagecraft. VR will have to find a new way. I suspect that for a significant while, cutscenes and the like, when ported into VR will end up simply reusing the language of cinema, and have to find ways of coping with translocation.

The easiest, as you allude to with the idea of 'watching on a cell phone' is to find a way to transition to a more screen like experience for the duration of the non-interaction. Perhaps the players view could slowly reduce in FOV and depth until they were looking at a more traditional screen shaped window of video that hovered in the center of their vision before the transition to the pre-set content, or perhaps there could be some kind of screen or display in the environment to which their attention is somehow drawn or even forced (although forced orientation changes can be just as disconcerting as translocation).

Simply playing the cutscene around them in the way you suggest with the 'ghost' idea might work, but it largely depends on what information is being conveyed. In such a mechanism you still have to have a way of drawing attention to small details that are important to convey, although this might be possible with good sound editing and and doing triggered events rather than a fully-non-interactive cutscene, but that in turn might depend on knowing you have really good positional sound available to the viewer, which the Rift itself doesn't guarantee.

The real answer is that there's no right answer, nor even an established set of techniques meant to act as a VR vocabulary of ideas in the same way that there is for cinema. It's a good question to ask, but in addition to asking forums like this for ideas, you should work on developing your own. Who knows, you might come up with something so innovative and groundbreaking that it becomes the 'Citizen Kane' of VR.

The main thing you loose when you do cutscenes on the Oculus is the ability to control the camera directly to set the scene you want to show. All the other parts carry over as long as you are smooth about the transitions to give people's brains time to adjust. The big change I think is that you need to change how you frame your "shots" instead of a simple rendered scene you need to now render a 3d enviroment and allow the user to look around. Once they are in this scripted cut scene you frame the shot by drawing the users attention to what you want them to look at.

Fix the camera position in the scene this is fine to do but do not fix the orientation. Then as the scene plays out use motion and sound to draw the users attention to the various elements as you go. If you frame it right I think they should almost naturally settle into the shots you want them to watch till something draws their attention away. You can still play with the orientation somewhat but it needs to be a very smooth and gradual change. Sudden changes are not good I almost made someone fall over by changing the camera on him while he was looking at my demo.

It is going to be harder to get right than a fixed scene but it allows for different possibilities the action that once took place off screen is now happening live in the scene allowing the watcher to pick what they look at. You just need to look at scene differently now it's not a collection of pictures its the entire scene with the user free to pick where they look.

Thanks for the responses; they gave me a lot to think about. This certainly does represent a paradigm shift in the art of storytelling.

Appropriately directing the viewer's attention will obviously be a key component to a successful scene. I think scenes in VR are analogous to the stage in a number of respects: both take place in a defined space (e.g. the stage); both seek to create a distinct environment within that defined space; both should direct the viewer's attention without controlling his or her gaze; and both make you slightly nauseous (kidding :D ). I will be thinking about this next time I go see a show.

In addition, it may also make sense, at least for some scenes, to include multiple points of interest. A scene in which the viewer's attention can be focused on one of three or four points while still conveying the essence of the story would be something interesting to experiment with. This may also add some re-watchability to a cinema/VR hybrid (I'm not sure if there is a name for a movie in VR).

Moving the view/player/camera around (not the orientation, just the location) will also be a challenge if used as a storytelling tool. I can't wait to get my dev kit and start experimenting!

Why wait start on your scenes now get things working on a standard monitor for the moment and use the mouse to look around. That way when you finally get the oculus you can integrate it which is pretty straight forward and get straight into the fun bit. This will also give you a really good idea of the differences between the two displays as you transition and adjust from something that looks good 2d to something that looks good 3d. Having a well known scene makes it easier to detect any problems in your Oculus setup and get those fixed really early.

just let go of control I believe is best, one of the best things about vr is you don't have to spoon feed anyone about what you think might interest them or be visually intriguing, and replay value is a total +1 with that view point, always seeing new things in a cut scene. many games have been using live action cut scenes for a while now, and I belive that is the way to go (as I hate with a passion those games who show trailers of this totally amazing graphics and the actual game is like, well won't give specifics we see it all the time, totally sub par to say the least). I don't won't to be directed, let me find something I find cool and share that on YouTube or something or share it with me), like others have said let the player look around in a cinematic, and if translation is required give queues for it or relevance or add special features where the player gets to view the hidden parts of the story to understand why things happened in the experience they had. a movie is static and locked down to a 2 hour(avg) experience, while games and now experiences are multiples of that. vr gives everyone the chance, like life, of being there own directors. I'd actually like to see a film made with vr in mind, that would be very moving If they look past throwing things at the audience.

"genetransfer" wrote:
just let go of control I believe is best, one of the best things about vr is you don't have to spoon feed anyone about what you think might interest them or be visually intriguing, and replay value is a total +1 with that view point, always seeing new things in a cut scene. many games have been using live action cut scenes for a while now, and I belive that is the way to go (as I hate with a passion those games who show trailers of this totally amazing graphics and the actual game is like, well won't give specifics we see it all the time, totally sub par to say the least). I don't won't to be directed, let me find something I find cool and share that on YouTube or something or share it with me), like others have said let the player look around in a cinematic, and if translation is required give queues for it or relevance or add special features where the player gets to view the hidden parts of the story to understand why things happened in the experience they had. a movie is static and locked down to a 2 hour(avg) experience, while games and now experiences are multiples of that. vr gives everyone the chance, like life, of being there own directors. I'd actually like to see a film made with vr in mind, that would be very moving If they look past throwing things at the audience.

I agree that this approach makes the most sense. As I alluded to in my previous post, though, there are certain storytelling tools that you lose (or are drastically different) with live action cutscenes. A perfect example is flashbacks. I'm not trying to create the VR sequel to Lost or anything, but I still wonder what new storytelling techniques may replace some of these "classic" methods.

A perfect example is flashbacks. I'm not trying to create the VR sequel to Lost or anything, but I still wonder what new storytelling techniques may replace some of these "classic" methods.

I don't think all will be thrown out the door just utilised differently. I think audio queues just before a flashback would help, just so the user is ready for the event when it happens, and just fade the scene to the flash back (star wipe :)).

people seem to not have too much trouble with being on rails in the rift, and I've yet to find out my self, but it really seems the issue is not being primary translate without control, it's when the camera looking control is lost and that appears to be jarring.
so while you may in the cut scene control the position and base rotation of the player head, the player is always able to look around while the story is in action. and the audio queue give the player an awareness that from this point till the end of the sequence they can let go of the gamepad or whatever and enjoy the story.

either way I think once more experimentations are tried, everyone will start learning from each other. I'm sure when more devs get the rifts and start sharing ideas, as is happening with the current user made content, we might get surprised at what people come up with that move beyond traditional story telling techniques.

If you want to see how to tell a story without cut scenes taking control of the players viewpoint, play through Half-Life 2.