“The First Time We Put The Camera Rig on the Technocrane It Went Right To The Floor.” Three Time Oscar Nominated Visual Effects Supervisor Pablo Helman Discusses His Pioneering Work with Martin Scorsese on ‘The Irishman,’ The Future of Artificial Intelligence in Visual Effects, and What He Learned From George Lucas When He Earned His First Oscar Nomination.

Three time Oscar nominated Visual Effects Supervisor Pablo Helman faced one of the most daunting tasks of his career when he tackled the de-aging challenge put forth by director Martin Scorsese for Netflix’s The Irishman.

The movie, shot from the perspective of an elderly Frank Sheeran [Robert De Niro], recounts his time as a hitman or “house painter” as it’s described in the film. Sheeran’s story is interwoven with Russell Bufalino [Joe Pesci] and the Bufalino crime family along with Teamster President Jimmy Hoffa [Al Pacino] over the course of several decades.

To appease Scorsese’s approach of no facial dots or green screen tactics, Helman became a pioneer in creating a revolutionary performance capture camera rig. This rig, appropriately named the “Three Headed Monster,” was the key to getting the long gestating film project off the ground.

Helman’s revolutionary work led The Irishman to ten Oscar nominations, including his own for Best Visual Effects. Awards Focus spoke with Helman to discuss the intricacies of his ground breaking camera technology, working through the advanced software and his views on Artificial Intelligence in filmmaking, and what separates Martin Scorsese from other directors.

Awards Focus: When when you first started working on the “Three Headed Monster,” did you start with the rig itself, or the flux software? How did those two come about?

Helman: The first thing was figuring out how we were going to capture the performance without the markers [dots] on the faces of the actors. We looked into software to see how we could leverage lighting and texture. When we were thinking about lighting and what we were going to need to capture, we knew we needed to get as much information as possible and the only way to do that is to get as many cameras on set as possible.

The tricky thing is that the only camera really respected on set is the director’s camera, any witness cameras that the visual effects team put on set are not taken into account. Therefore, the only way for us to get those cameras taken care of is to attach them to the director’s camera.

Camera Rig on The Irishman

AF: What was the early set up like for the director’s camera?

Helman: The beginning of that rig was just three RGB cameras — a digital camera in the center for the director, and then two digital cameras on the side. We realized as we were developing the software that it did best if the light on the actor’s faces was even, but that never happens on set, because there are lights from all over the place and they have harsh shadows. So we thought that maybe we can equalize the light and get rid of the shadows in a different spectrum in a way that wasn’t going to handicap the director of photography or the director.

AF: Which was through infrared?

Helman: Exactly, because the human eye cannot see infrared light. We switched to infrared on the witness cameras and put several infrared rings in front of the lenses so that we could show infrared light on to the actors. Then, software picks that up and and computes the infrared light to get us the performance in terms of geometry.

AF: Was the rig easily mobile or did adjustments have to be made there? Did it ever get damaged at all?

Helman: The three cameras in the rig had to be film grade cameras, so that already gave us a certain poundage because now the three cameras have to be mounted all together and communicate with each other so that means an awful lot of cables. Two weeks before the the principal photography started we had a rig that was 84 pounds. When we put it on the technocrane, the technocrane went to the floor [laughs].

We spent the next two weeks with Rodrigo Prieto, the DP, taking things out until we shed about 20 pounds. So when we started Principal photography, the rig was 64 pounds. But the technology is advancing so fast, right now I am working with that rig on another show that has half the weight of the one on The Irishman.

Joe Pesci Before and After

AF: When I when I was watching the film, I thought Pesci looked, I guess, the most flawless. Do you think software and “three headed monster” worked best on him?

Helman: To me it was the same for each actor, but it’s funny that you mentioned Pesci. I think one of the things is that people don’t realize is that you see Pesci at around age 53 in the film, he was never that thin during his 50s. Marty made the decisions to keep him at his current size throughout the film. Also, Pesci hasn’t been acting for the last ten years or so, which adds a freshness to his appearance in the audience’s mind I think.

AF: The film spans obviously several decades, at what point did the film start relying on makeup rather than the Flux software?

Helman: I think for Robert De Niro and Joe Pesci it starts around the casino sequence, so when they are about 60 or something like that. Pacino never goes into makeup, his is all through CGI. computer graphics. We were going to do makeup but Pacino had an allergic reaction to something so he couldn’t do makeup.

Flux Technology Used on The Irishman

AF: This film is such a beautiful tragedy, particularly through the eyes of De Niro’s character. Even though he survives throughout the film, he ends up alienating his family and living a hollow, lonely life in a nursing home. What do you think about the film?

Helman: I think it’s a great thinking piece, the movie is about many things. It’s about the passage of time. I think that it has a lot to do with the point of view in which the film is being told. So you have Marty Scorsese, who has had an incredible life and an incredible amount of knowledge and experience. Marty sees things from his 77 year old director’s eyes. On top of that, you have the whole movie being seen from an 83-year-old character’s eyes. People can take a lot of things away from this film, just because it’s a mafia film doesn’t mean its themes are limited by the genre.

AF: Working with Scorsese is a dream for most aspiring creatives. What do you find unique about his process and what do you find interesting about him as a person?

Helman: Well, I think he’s an encyclopedia on film. I have worked with many directors, and sometimes even with films that I didn’t worked on, the director’s work feels like an imitation of something, and, that to me, is missing the process.

You realize when you work with these masters like Scorsese, that everything that they do is because it belongs in the story. They’re trying to tell a story in a specific way and the camera moves in a specific way, and it ends up in a specific way, and the framing is such that it It tells a specific story.

The Director doesn’t have very many ways to tell a story, right, he has a camera, you point the camera towards something and he has light, and he has framing, and acting and that’s about it. It’s up to the directors to point the audience to take a look at something specific. He’s also very exacting, and he’s very peculiar and particular about what he wants to say. He’s a master in communication.

AF: Your first Oscar nomination came from working on Episode II of the George Lucas Star Wars saga. What was your experience like with Lucas and what surprised you the most about his process?

Helman: What is it they say, “A long, long time ago?” [laughs]. It was almost 20 years ago when I started working with George. I saw him everyday for three years and it was great getting to understand why he makes decisions that he makes. I learned a lot in terms of editing from him because he’s basically a director where everything happens in post for him. He has an incredible sense of storytelling. So I learned a lot from him in that sense. A little bit of every director just stays with me, especially if I work with them for some time. I treasure that experience.