Establishing Presence in Narrative 360 VR movies:

Recently there was an interesting discussion on Facebook’s 360 filmmakers forum – Yes, Facebook is not about selfies anymore. It was about the history of VR cinematography, (a compelling read), but it quickly came down to one point – the pros and cons of 2D 360 versus 3D 360.

Comparing 2D 360 to S3D 360:

One of the arguments made in the facebook discussion thread was that of audiences not caring whether something was shot in 3D 360 or not – The claim being that audiences are wowed sufficiently by VR itself, to not care what the medium (2D or S3D) was, but simply (if) how great the content was.There are two camps on the subject, and as a disclaimer, I subscribe to the S3D 360 school of thought and practice. Without getting into lengthy back-story, I’ve copied below, my perspective (unavoidable bad puns will abound) on the subject.

I believe the only way to make a fair comparison, and keeping narrative films in mind, is to show the same scene in 2D and then in 3D.

Here’s one rough example that was on my laptop, so let’s make do.

Both images need to be seen on a VR headset – the Oculus Rift is a good bet and the free Kolor Eyes viewer is a good choice (choose Over/Under in Koloreyes’ Oculus rift menu)

Obervations; 2D image – download it by clicking the header image in this article.

The headboard is plastered to the wall.

That is one huge Headboard, and those are some big *ss pillows.

Am I (you) sleeping on a mattress on the floor, or is it a bed?

Where does the bed end and the balcony door begin?

The ceiling lamps closer to the headboard – are they decals? or real lamps.

How about the ceiling lamp pair further away? (Hint: Do their shadows lend depth cues)

The angular cuts/shadows of the passage-way and the wardrobe – Do the shadows lend depth cues and thus offer the scene some sense of depth?

What I’ve learned from studying various such scenes are:

I need to remove my socks when jumping into someone’s bed…no, seriously…

If I were making a 2D 360 movie, a good DP would be indispensable. (S)he would need to make sure depth and scale is being conveyed in VR either via lighting/shadows/contrast and, or, shallow focus etc.

I need to be in Hollywood, get a few million in funding, and build a 3D-360 rig that would give me better resolution than in the next image.

How hard is it for someone to come up with a further refinement of the good old Si2k Mini stereo rig to create a back to back 360 capture solution? (The no-stereo-at-the-edges in fish-eye lenses argument doesn’t quite hold, in narrative film-making)

The 3D Version – Cinematography to establish Presence in Narrative S3D 360 movies:

a) Suddenly we can start flirting with that term “presence” (O.K. there are some seam lines that need clearing up, and i’m not an aftereffects expert)

b) We also abruptly realize that my (your) legs don’t really reach all the way to the passage / corridor entrance – We have a sense of scale when stereoscopic cues kick in.

c) This is a narrative scene, so hopefully the acting & actress, who will be part of the finished shot, will keep the audience’s attention hooked to where the director wants – in the center of the room – and long enough, that the only seams they’re looking for aren’t on the walls.

I’ve shown many such scenes to people, and the results are as expected. It really is a night and day difference between 2D 360 VR and S3D 360 VR.

It’s worth repeating that in conducting any fair test of audience appreciation of VR as experienced with headsets, comparisons have to be made using the same scene in 2D-360 and S3D-360.

People don’t know the difference – true, and that’s only because they’re being shown nokias…then an iphone comes along.

Observations:

2D-360 is easier to produce.

For Vista shots (Grand Canyon, City from atop the Eiffel tower, Huge larger than Life heritage architecture) …2D 360 is spell binding.

2D 360 is degrees (sorry) easier to stitch than s3d-360



Where 2D 360 doesn’t hold a candle to s3d-360 is:



1) Sense of scale – If one is aiming for “presence” and immersion in VR – you can’t fudge giant Rollerskating people around you as affording ‘VR presence” (check out one of the videos on Samsung’s MilkVR site.)



2) Any shot that is not “documentary” in nature, and specifically not of vistas, pulls you (me) out of the scene.



3) I don’t think a 2d-360 movie would do very well if shot from the conventional height of an actor. – In VR (and s3D) you can’t cheat scale, or depth.



4) A lot depends on the “angle” of objects/subjects in a scene and if they cast shadows – that is the only way to judge “distance” in 2D 360. Stereoscopic/Binocular spatial cues are non-existent. – it’s a wallpaper globe world.

Motion parallax as a sensory cue – has it’s own set of problems as 2D 360 filmmakers depend on muticam 360 rigs built with cameras that are (yet) un-synced and also exhibit jello-cam (rolling shutter) problems.

Even if these problems were fixed (via global shutter CMOS/CCD sensors and Machine vision cameras with genlock sync) the dependence on stitching multicam footage (with even moderate movement) is not error-free. – This last point holds true for s3D 360 too, but motion parallax as an argument for enhancing “presence” in 2D-360 still does not mitigate points 1,3 and 4, above.

Where I would use 2D 360 is for drone cam coverage of events / visual documenting of larger than life structures. Where I would use s3D-360 would be for virtual “telepresence” – teleporting the audience from a dusty battlefield to a bedroom as the narrative unfolds.

If there’s anything to be taken away from this essay, it is that – just as Palmer Freeman Luckey showed what a VR headset can be… so too should 360 camera manufacturers and solution providers strive to create solutions that lend as accurate as possible a sense of presence in Cinematic VR.

My focus is on narrative S3d-360 film-making, and to that end, I still feel the Director or Storyteller should can be in control of the 360 “frame”. Not having positional tracking, in my opinion, does not / will not detract from the audience experiencing a good degree of ‘presence’, in a properly scripted, and executed s3D-360 movie. Headtracking (Pitch and roll yaw) is enough, because Spatial depth is a formidable “presence” cue, and when augmented with other monoscopic cues, makes for truly “immersive” visual storytelling.

– The new meathook S3D rig i’ve got has a lesser nadir hole than what shows up in the example shot above, and will also be feathered better.

Scenes used in this article are from “MAYA” – a Real Vision 360 VR motion comic currently under production. Produced by: Master Media.