Respecting the Spatiality of a scene in Virtual Reality:

When Directors and Cinematographers start responding to Facebook’s call to Hollywood for Immersive 360 movies, one important aspect could get overlooked by film makers.

Decades of 2D movie making experience of creating depth in a scene via lighting, lens effects, and framing, will need to be un-learned. In 2D it’s easy to “cheat” depth.

All this changes when getting into 3D look-around VR film making. For example, one does not need a montage of different angles (cuts) to represent the geography and spatiality of an environment in VR, as is needed in a 2D movie.

Also, if the film/movie is captured in Stereoscopic 360… The one thing that 3D excels at, is in recording and presenting spatial depth in a scene.

(image credit: The BBC’s Oculus Rift project Click for larger)

Crafting a VR depthscape: The Immersigrapher.

Look at the first image in this article; the man sitting on the bench (click for larger). This is a rough previz story board done in the excellent previz software FrameForge 3D. Of interest to us is the Camera data at the bottom of each panel. It shows Camera height at 2.91 meters from the ground. In a Virtual Reality film – think of this number as the “neck” height the Oculus Rift (OVR) Camera is placed at in the VR world.

What is the Motivation for placing the Camera at 2.91 meters?

This is the first question an “Immersigrapher” has to ask a Director or Cinematographer, while they frame a VR scene. It may well be that the shot is not married to the script, except to show an establishing shot of a train station with a man on a bench reading a newspaper. In that case it’s OK to place the camera at such an arbitrary height.

However, what if the movie is meant to “afford presence” – one of the buzz phrases in Cinematic VR filmmaking – and the script reads:

“The detective approaches a man sitting on a bench reading a document, and comes closer to have a glance at what the man’s reading”. We should be aware that in 3D Virtual Reality, the spatial dimensions of a scene are faithfully recorded and presented, so the VR camera placement at 2.9m above ground would yield a POV of the detective running a risk of seeming like the detective is a 3 meter tall man!

Things get further complicated if in an effort to preserve “stereo roundness” or for other reasons, the interaxial (Stereo 3D camera base) leads to hyper stereo and thus “miniaturizes” the seated man, bench or the rest of the props in the scene.

In a 2D film, this is not a concern as depth and perspective can be cheated. In Stereo 3D, a film maker has to step back, take a macro look at the scene and then decide on what the motivation is for a particular choice of camera placement.

And… in a 360 Virtual Reality scene, where the audience might not be undergoing a HeadTracker Hijack (HTH), but instead have the freedom to look around the scene, Gigantism (the VR headset wearer feels like a giant) might pull the wearer out of his/her suspension of disbelief.

The image above from the movie Drive Angry 3D, shows another side effect of un-motivated Camera placement; Perspective mismatch and Depth Continuity issues between this shot and the next.

[youtube id=”aeW37ZgbwYg”]

(view in 3d on youtube)

Motivated Camera movement in Stereoscopic 3D:

The movie Hugo, has received well deserved acclaim for showcasing stereoscopic 3D as a new medium of visual storytelling. There is much to be learnt from the camera work and stereoscopy in the film.

Some observations from the 3D Trailer above:

0:30 – Notice the perspective, height and point of view as the camera tracks into the scene.

0:43 – A perfect combination of interaxial setting (no miniaturization of the scene or king-kong effect) noticeable as the camera cranes up in the rise-up shot.

Jaunt VR Rig (image credit: No Film School)

Challenges in Designing an S3D VR Camera Rig:

Careful orchestration of a VR Camera move (correct interaxial, timed cut, taking the proper amount of dwell time into consideration) will lead to compelling Virtual Reality Cinematography in scenes involving Jibs and Crane shots. Of course, in simple Mise-en-scene VR cinematography, it gets easier as there are less variables to look into. It’s easiest when creating a VR movie completely in CG or even via a Realtime Game Engine.

Things get very interesting and challenging for live action look-around 360 VR cinematography. One of the state of the art 360 VR rigs seems to be the JauntVR rig pictured above. There is no known 360 live action rig to date that might allow for FIZ (focus/iris/zoom) like control of the multitude of parameters when shooting synchronized 360 VR.

VR filmmakers will discover how to use HIT (Horizontal Image Translation) to balance depth volume of a 3D-360 scene, currently it’s only a handful of astute 360 3D filmmakers who know how (Realvision being one of them.) But even then, it’s for depth sweetening of the entire 3D volume in the scene.

Examples of Camera Motivation:

If an actor in a scene is riding an elevator going up, or a Helicopter lifts off from a helipad with an actor looking down, this would be the motivation for where the camera is placed in S3D Virtual Reality. Ultimately the premise of Motivated Camera movement for effective stereoscopic 3D VR movie making is the choice of the Director and the DP. They have to have the ability to think in 360!

To see how Crane and Jib shots could break the suspension of disbelief for audiences and pull them “out” of the scene (in a regular Stereoscopic 3D movie) pay close attention to these scenes from:

1) Transformers: When the main actor comes out of his parents Mobile Home.

2) Drive Angry 3D: A high shot looking down on the gun fight in the motel room. – Side note: in S3D, bigger is better. So a cinema sized screen does save the scene to a certain extent from extreme miniaturization. A different choice of interaxial (lesser) and lens, would have made for a better 3D scene, but if this scene was shot in Stereoscopic 3D, look-around VR, the audience would feel like a giant spectator watching a diorama unfolding in front of them.

This article is not a set of rules, but more a guide, aimed at seeding ideas as we move forward toward defining a new language and grammar for the art of Virtual Reality Filmmaking.

Presence in VR 360 films:

A related essay on “presence” in VR films is :here: