Augmented Reality For Filmmaking

Anybody​ who’s watched a behind-the-scenes feature for a post-Matrix blockbuster has probably seen this scenario: actors and actresses on a green screen, talking to a tennis ball on a stick or to a person in a tracker-studded leotard. What the director sees behind the monitor on-set isn’t much better. Soon that might not be the case.

Last year we highlighted this video introducing the “Blackbird,” a creation of The Mill. The Mill’s a prominent SFX and post-production house, and the Blackbird alone is cool tech — it can serve as a physical stand-in for a car, letting car ad makers put the appropriate photo-real render over the Blackbird’s chassis in post. This cheesy new Chevrolet ad depicting a race between a human and a self-driving sports car uses the Blackbird, but it’s got more than one trick up its sleeve:

One bigger, fresher secret behind this ad is that the directors were actually able to visualize the CGI cars over the Blackbird as they filmed. The Mill could pull this off thanks to augmented reality tracking and real time rendering. The final film was made using Unreal, a video game engine that powers titles like Gears of War 4, to do the effects. This means that the final product isn’t a static film — at the ad’s Game Developer’s Conference premiere, you could change the color and model of the car in real time, while the video played.

Real time effects could be huge for interactive media, giving fine-grain control over what’s on-screen to viewers. Cooler still is the freedom granted to filmmakers doing effects-heavy films — shots that previously would have been painstakingly storyboarded or assembled entirely in post-production could be improvised on in real life thanks to augmented reality visualizations.

Still, as movies like Mad Max: Fury Road remind us — there should always be a place for good old-fashioned practical effects in film.

Speaking of self-driving cars: check out this simple demo that lets you play with self-driving cars in your web browser.