Mr. Fincher said the last thing he wanted was for an audience to seize on period details like an avocado-colored rotary phone, or an actor’s sideburns, and miss the point of a scene. In several days on the set in San Francisco and Los Angeles in late 2005 and early 2006, he could be seen constantly retaking shots to dim a lamp, remove a too-colorful car, or alter the costume of an extra whose garb seemed lifted from a fashion layout rather than what people really wore.

Mark Ruffalo, who stars as the lead detective, said “Zodiac” was unlike any other Fincher film. “He’s just completely gone for the character and the story, and has sort of made that the rule, and not the look,” he said. Near the end of filming, Mr. Ruffalo recalled, Mr. Fincher said he’d watched a rough assemblage of about half the movie. “He said: ‘I think it’s great, but I’m in territory I’ve never been before. I just don’t know if they’re going to get it. And that’s exciting news: ‘Here’s my brand, and I’m stepping outside of it.’ ”

Image Mr. Fincher and Mark Ruffalo on the set of the film. Credit... Merrick Morton/Paramount Pictures

More difficult was changing the way Mr. Fincher worked with, and made demands of, his actors. On “Panic Room” he grew frustrated with his process — detailed storyboarding and previsualization to diagram a movie shot-by-shot — because it left little room for discovery, Mr. Fincher said. “It just felt wrong, like I didn’t get the most out of the actors, because I was so rigid in my thinking,” he said. “I was kind of impatiently waiting for everybody to get where I’d already been a year and a half ago. And I’ve been trying to nip that in the bud. I felt like I needed to be more attentive to watching the actors.”

He added: “Every once in a while there are actors you can defeat.”

For Jake Gyllenhaal, who stars in the movie as Mr. Graysmith, Mr. Fincher’s attentiveness was a mixed blessing.

Mr. Gyllenhaal said he came from a collaborative filmmaking family: “We share ideas, and we incorporate those ideas.” He added: “David knows what he wants, and he’s very clear about what he wants, and he’s very, very, very smart. But sometimes we’d do a lot of takes, and he’d turn, and he would say, because he had a computer there” — the movie was shot digitally — “ ‘Delete the last 10 takes.’ And as an actor that’s very hard to hear.”