Compositing elements in Virtual Reality:

The image above, is the result of over 3 weeks of research and testing in Cinematic VR film-making. While it is not perfect, we are in the midst of stirring up some secret sauce to enable a smooth workflow, for Maya the 360 motion-comic.

Compositing in 360 space is not trivial. In fact, as of today, NO mainstream compositing software or solution exists for compositing in 360, in 2d. In Stereoscopic 360, compositing is a whole different beast. This knowledgebase will be updated during production of the VR novel.

For now, a few pointers to keep in mind:



(click for larger)

In S3D, you can’t cheat Scale. – If you’re shooting a scene from the point-of-view of an adult, make sure the 360 camera rig (or Stereo 360 rig) is at the eye height of an adult.

If you download the above image and view it via a VR HMD, notice how you will be looking “up” at the characters. Good for a seated VR experience, but if this were a scene where the Director would like ‘you’ to be a voyeur, listening in on a private conversation between two people at an airport… the vantage point would be suspect. It would be strange for you to be at crotch level!

(Trivia: this scene was shot Guerrilla VR style at an airport in Stereo 360. Compositing in AfterEffects)

(Trivia: this scene was shot Guerrilla VR style at an airport in Stereo 360. Compositing in AfterEffects) Try to match your CG camera as near as possible to the lens that shoots the 360 video/panorama. For the image above, the CG camera that shoots the girl is set at a 14mm focal length.

Pay no heed so-called VR Gurus. There’s a few of them cropping up who claim to know everything about Cinematic VR production. While best practices are great to adhere to, it’s still an open playground, and the Language of 360 VR storytelling is still being written. There are no experts, yet.

** Update** Aug1st 2015

As this article has been receiving hits from many AAA studios (Hollywood, Asia and Australia)… We’ve decided to make known some of the sauce currently in use, and other solutions for compositing in Equirectangular space

1) Sat Domes’ Camsphere

2) Mettle’s Skybox

3) Exclusively for Mac (in beta) : Dashwood Cinema’s VR toolkit

4) The not-so-cheap but very effective – Navegar Full Dome (also does equirectangular) plugin for After Effects.

5) Soon to be released for Nuke: see video below

Observations:

In a VR film, if seen from a first person P.O.V, a unique problem arises – if the wearers “Body” is superimposed or composited in place under the actual 360 camera, what happens if the wearer turns around? VR exorcism!

One possible solution to the problem above, is to have a dynamic “headless avatar” generated by VR movie playback software. This avatar could then respond to head-tracking and accelerometer/compass heading.

If you’re interested in some of the results of compositing in stereoscopic VR, grab a copy of the Stereoscopic 360 teaser of Dirrogate: Memories with Maya, and avail of an early access discount of 50% on the final VR novel when it launches.