“Even if you're not into sea urchin, you would probably dig this restaurant,” Justin Theroux says, as if he's letting me in on a secret. “Everyone knows Jiro Dreams of Sushi”—the Netflix documentary about the slavishly devoted sushi chef Jiro Ono—“but this place is billed as the place that Jiro likes to go.”

It's hidden where all the best places in Tokyo are: through a back alley and past an unremarkable building that leads to an unmarked door. And the sea urchin. Oh, man—the sea urchin! The dish requires surgical precision from the chef: The spiny shells are split open; then the meat is scooped out and whipped into a rice that's been vinegared and cured for a few days. The whole thing is garnished with a couple more ounces of the spineless creature for good measure. “The trick to that place is giving you just enough to make you want it so much more,” Theroux says fondly. It's a dish so good you want seconds, thirds, fourths, and fifths.

You see, Theroux loves Tokyo. Not because he has his familiar spots, his favorite hangs, or the one restaurant he always goes back to. It's just the opposite. “It always feels like it's too brief, being in Tokyo. The draw that keeps you going back is that it's a very hard place to tack down,” he says. “Tokyo is an organism that keeps growing and evolving.”

As a city, Tokyo makes absolute sense to him. When someone has a runny nose, they wear a surgical mask instead of greeting you with a hug. The 7-Elevens have outlines of footprints on the floor to shape the polite, orderly queue of people waiting for egg sandwiches. There are rules everywhere, but if you give in to them, you're “like a leaf in a stream,” he says.

“I'm glad I'm not remembering the names of a lot of these places, because they'll immediately get put in a Time Out guidebook and then become savaged by tourists,” Theroux says. “It ruins the place you went to. You can't go back to it again, because you're like, ‘Oh, it's now a bunch of Australians.’ ”

When Theroux returned from his trip, we got him on the phone and asked him for the stories behind the many stops he made alongside photographer Mark Seliger.