Martin Scorsese needed a favor. Well, not a favor, exactly—it'd cost quite a bit of money. He needed to make Robert De Niro young again. It was fall of 2015, and Scorsese, knee-deep in production on Silence, was having Thanksgiving dinner in Taiwan with Pablo Helman, one of Industrial Light & Magic’s veteran visual effects supervisors. At the time, Scorsese had been prepping a Frank Sinatra biopic, and he and Helman were talking shop about how to have one actor play a person at many phases of their life.

“He’s a very curious person, and he said to me, ‘Talk to me about that,’” Helman says, reflecting on that first meeting in one of ILM’s San Francisco offices overlooking the Presidio. “So we talked about how to make somebody younger. He said, ‘You know what? I’m not going to do Sinatra, but I’m going to do this other thing.”

That other thing turned out to be The Irishman, Netflix’s 3.5-hour, decades-spanning adaptation of I Heard You Paint Houses, Charles Brandt’s book about the alleged mafia hit man Frank Sheeran. Scorsese sent Helman the script, and he read it overnight, all 170-ish pages. “We were shooting [Silence] the next morning,” Helman recalls, “and I said, ‘I’m in.’”

What he was in for was a four-year quest to reinvent the way Hollywood makes its stars (not) act their age. In recent years, the race for viable de-aging technology has ramped up. Studios used to just ask VFX houses to do a little facial touch-up—thinning out wrinkles or fixing bad makeup—but as computers got faster and software got better, they started realizing fuller-scale de-aging was possible. Back in 2008, for David Fincher’s The Curious Case of Benjamin Button, Brad Pitt’s facial expressions were captured using the Mova camera system, which places a series of cameras around an actor to gather all of their facial movements and then uses that data to digitally build someone older/younger. (Since Benjamin Button ages backward, Pitt didn't need to be de-aged so much as he had to be re-aged.) For this year's Gemini Man, directed by Ang Lee, the VFX team cut Will Smith’s age in half by scanning his face, building a database of his expressions, and merging that with data from earlier performances—Fresh Prince of Bel-Air, Bad Boys. The 49-year-old Smith did his acting on a mo-cap stage, wearing headgear and face-tracking dots while cameras captured his every move; that performance was then combined with the information in the database to re-create a 23-year-old version of the actor.

To make Irishman, though, Helman couldn’t replicate Weta Digital's process on Gemini Man. Scorsese wanted to be able to shoot his movie the way he would any other. No mo-cap. No actors walking around in headgear. “The film takes place from 1949 to 2000, and it goes back and forth in time continuously,” Scorsese says. “The problem is, by the time I was ready to make the film, Bob De Niro, Al Pacino, and Joe Pesci can no longer play these characters younger in makeup.” So, when Helman said he could do it more au naturel, the director was intrigued. “I said, ‘I don’t know. I can’t have the actors talking to each other with golf balls on their faces. It gets in the way of the actors, and the kind of film this is, they need to play off of each other. If you can find a way to lessen the technical aspects of it, it could work.’”