It was early July, about nine weeks before the debut of The Late Show with Stephen Colbert, and we were sitting in his temporary office above a BMW dealership on the far west side of Manhattan. He looked very tired, and he was apologizing (unnecessarily) for rambling on in a way that was maybe a little uncomfortably overemotional. “I didn't leave the studio until 2 A.M. last night,” he said. “Didn't get to bed until three, and I've been traveling and just got here—.”

He'd been up late doing a strange stunt the night before, stepping in unannounced as host of Only in Monroe, a local public-access program in Monroe, Michigan, about forty miles south of Detroit. There was all sorts of pressure on their first show, he said. “First show! First show! Well, fuck the first show. There's going to be 202 this year—how do you do a first one? So I just wanted to go do a show someplace. And now we've done it.”

The idea was to do Only in Monroe more or less as it always is—same production values, same set and graphics and crew—just a ton more jokes. His first guests were the show's regular hosts, Michelle Bowman and (former Miss America) Kaye Lani Rae Rafko Wilson. (Colbert on-air: “I'm not sure how many people that is.”) He did Monroe news and the Monroe calendar, and about twenty minutes in, he brought out his next guest, “a local Michigander who is making a name for himself in the competitive world of music, Marshall Mathers.”

SEE MORE: How To Suit Up Like Stephen Colbert

We were talking about the logistics involved in pulling off something like this, and how great it felt for him to be improvising in front of a camera again, and the curious tensions that popped up in his interview with Eminem. And then we got onto the subject of discomfort and disorientation, and the urge he has to seek out those feelings, and from there it was a quick jump to the nature of suffering. Before long we were sitting there with a plate of roast chicken and several bottles of Cholula on the table between us, both of us rubbing tears from our eyes. “The level of emotion you're getting from me right now—I'm not saying it's dishonest,” he said. “I'm just saying it's not normal. I'd really love to go to bed. I promise you, I do not spend my time on the edge of tears.”

I've easily played the recording of that conversation a dozen times, only one of them in order to transcribe. And while we spent plenty of time talking about comedy and the conventions of late-night and the sheer practical challenge of doing a show twice as long as his old one—the thing I've been thinking about the most since my time with Colbert is loss. The losses he's experienced in his life, yes, but really the meaning we all make of our losses. Deaths of loved ones, the phases of our children's lives hurtling by, jobs and relationships we never imagined would end. All of it. Among other things, our lives are compendiums of loss and change and what we make of it. I've never met anyone who's faced that reality more meaningfully than Stephen Colbert. I suppose, more than anything, that's what this story is about.

Also: ball jokes. Or the absence of them. They're doing network now, after all, and Colbert has declared a moratorium on ball jokes. (I believe I was present for the last one. It involved Greece and the Eurozone—and Paul Krugman's balls.)

They did the public-access show live at midnight, with no advance publicity and no Twitter or Facebook posts afterward. The only way the world would ever know that it happened is if someone, an insomniac or an inmate or one of the show's twelve viewers, looked up at the screen at some point and recognized Colbert hanging out with Eminem next to the potted plant. Maybe that person would tell somebody, and maybe that other person would tweet about it.