Built 90 years ago, the Ed Sullivan Theater in New York was has lived through many acts. First as a home for Broadway plays, then a nightclub, and eventually a broadcast studio for CBS. Over the years, shows have come and gone, hosts exited and new ones brought in. For its current incarnation as the backdrop for the Late Show with Stephen Colbert , veteran production designer Jim Fenhagen peeled back all the layers in the space and then added a medley of details that would make this chapter in the venue’s life a visual feast.

Fenhagen is no stranger to the world of television, designing sets for CCTV, Piers Morgan, the Colbert Report, and the Daily Show, among others. The company Fenhagen works for, the brand experience agency Jack Morton, has earned about 20 Emmy awards for its work over the years.

“There is a quest for coming up with something either original, iconic, or one-of-a-kind,” Fenhagen says of his process. “The quest for that is always in my head when I’m doing a show—something for that one show that no one else has.” Additionally, the space has to represent Colbert’s personality at a glance.

Colbert is very involved with production details, and for the Late Show he thought it was important to celebrate the theater, which is a landmarked building. “The challenge for us was, how do we embrace this theater, but at the same time create an intimate space for Stephen to have a one-on-one conversation with guests?” Fenhagen says.

Fenhagen and his team did away with a lot of the bulky equipment and paneling that obscured the Neo-Gothic detailing inside. They also installed a monumental dome—complete with inlays of Colbert’s face—in the style of the original building’s design, and constructed a multi-story set for that would bring a sense of scale to the soaring space.

“Stephen told me, ‘The theater is grand and huge but it’s too big for the jokes—jokes need to be contained,'” Fenhagen says. “So we had to figure out a way to contain the jokes.”

There are two audiences for the Late Show: people in the theater and the millions of at-home viewers. While cameras can pan around and make tight shots for television, the same isn’t true for the live experience. To make the set feel more comfortable, Fenhagen called for smaller spaces to be built on the stage. A balcony wraps around the desk and interview space and creates a ceiling for the main floor. The idea was to step the eye down. To represent Colbert’s intellectual side, Fenhagen suggested a double-height library whose shelves are adorned with a few props from the Colbert Report.