Industrial Light & Magic’s digital models supervisor for the film was Bruce Holcomb, a visual effects veteran who’s been part of every Marvel film the company has worked on going back to 2003’s Hulk. Heading up the digital model shop, Holcomb and his team were responsible for creating everything from the bricks that Iron Man and the Hulk tear up from the city streets to vehicles like the Quinjet — but their most important work was on Ultron himself. I spoke with Holcomb to find out how ILM used Marvel’s designs and James Spader’s motion capture performance to create the metallic monstrosity.

Joss Whedon’s Avengers: Age of Ultron cleaned up at the box office this weekend, scoring the second-highest opening weekend of all time (the original Avengers is number one). While audiences certainly want to keep up with the ongoing saga of Iron Man, Black Widow, Captain America, and friends, a large part of the appeal this time around is the titular villain himself: a wise-cracking robot who’s got humanity’s extinction on his mind.

Bryan Bishop: Between Spader’s work and the various iterations he goes through, Ultron has a lot of different facets. Can you walk me through the process of creating him? Bruce Holcomb: When we first heard about James Spader playing the role, it was extremely important to Joss that we give him a robot that can perform all the nuances they were capturing from the motion capture. Initially they sent us different takes of Spader kind of walking around in a circle and acting out some parts, and he has this kind of weird head-tilt thing that he does, that’s very unique to his acting style. So when we first got the artwork [Marvel] had already drawn a guideline of what they were hoping for, from the menacing aspect of him. And then there was another wave of finalized artwork that came, that was supposed to be able to do all the nuances and expressionary things that Spader did, that we started the process on. …They gave us a really good initial model to start with, and then from there it was back and forth, pretty much all the way up until nearly the end of the film. I would say 80 percent of the design that was implemented by us here at ILM was in the facial area and then how he was going to move mechanically, based on the performance that Spader was giving. "It was extremely important to Joss that we give him a robot that can perform all the nuances they were capturing." So the task was almost two-fold. You had to capture the experience that he was performing and then the complexity of what they wanted to see with the robot, interior-wise. They kept mentioning a "watch within a watch"-type technology. …And then we implemented that, and we started putting him into shots, where there were many issues with his flexibility and stuff colliding. He couldn’t do a full range of motion like they were hoping for, so we had to solve a lot of problems here on our side. And then it kind of moved up to a fourth stage, to get everything to work together to where Marvel was happy.

He evolves a lot over the course of the movie. How did those different designs develop? The robot, the way it was described in the story initially to us, is that it was actually Baron Von Strucker’s robot. It didn’t have any Iron Man technology in it. So the first time that you see him is actually part of Tony Stark’s group of robots called the Iron Legion. And that’s that kind of spooky, wiry character that comes out when they’re at the party sequence. But then when he transfers his intelligence over to the other robot, which we called Ultron Prime, it’s a whole different robot not based on Tony Stark’s technology at all. And then he creates these things that we called "sub-Ultrons," which are basically the minions that run around and serve the only purpose of being blown up and destroyed. "He looked like a knight in armor. … He’s bigger, stronger, scarier." Those were different versions of him, lesser versions, but we didn’t go back and forth between Ultron Prime and those versions at all. They didn’t share the same topology. They shared a similar look, but not any of the same forms. And then when he encases himself in vibranium at the end, it was almost like he was not really making new plates that fit on top of his plates, but he was exchanging his armor for this vibranium armor that had more of an elegant feel to it. In the end, to me, he looked like a knight in armor. … He’s bigger, stronger, scarier. He had a lot of points on him, like a claw feel. He’s also pretty tall — certainly taller than Spader. How do you deal with that differential and make eyelines play? You had Spader on an apple box for a lot of the shots, and then you had this guy walking out with a stick with a big red dot on the end of it where all the actors could look up to and act to. And then the rest was just filled in by us. There wasn’t a specific formula to how we pulled off his height exchange. Of course, I think in shot-specific stuff we’d scale him up or kind of just make him fit in the shot to make sure everything was jiving right. There wasn’t a real formula to it, so we just played that one by ear. What are you favorite Ultron moments in the movie? Are there any particular scenes that you watch and say, "We really got that one right?" I would say when he’s first introduced. I was particularly proud of how that was handled, because a lot of the times we’ll work on assets and we’ll really get to a point where we’re trying to show off a lot of the mechanisms and things that we build into him. But I felt Ultron’s main thing was that he was dark and kind of a scary character. So when you play him in the dark aspects of the film, where there’s not a lot of light — there was a lot of specularity to a lot of the detail that we made, that you can kind of catch. If you go back on different viewings of the film, you can catch how much complexity is in Ultron the more times you actually stare at him. The first time you’ll see it and go, "Wow, that’s pretty cool," and you’re watching the performance, but on different viewings, you can look at his interior geometry as he’s just walking around.