AMC’s Mad Men has been called “the greatest TV drama of all time.” And for the last five seasons, Jon Hamm and Co. have worked hard to live up to that reputation, with increasingly darker and more intense story lines. But fans of the show know there’s a lighter, funnier side to the series, too, in characters like Harry Crane (Rich Sommer), Ken Cosgrove (Aaron Staton), Stan Rizzo (Jay Ferguson), and Michael Ginsberg (Ben Feldman).

Us Weekly sat down with the foursome — appropriately dubbed the “Casual Cool Quartet” on the magazine’s list of Hot Hollywood Style Winners — to get the scoop on their wardrobe secrets and on-set antics. (Spoiler alert: Pants play a major role in both.)

“My personal style is almost completely absent from my life,” Ferguson jokes to Us. “I spend an uncomfortable amount of the day in my robe.”

“Let me tell you how it works,” Sommer explains. “You start with a T-shirt. Over the T-shirt, you put a shirt that usually has buttons. And then maybe a sweater. And that can have buttons or not buttons. Because that’s where you make the most decisions…And then pants. Usually pants.”

“I owned one suit before doing Mad Men, and it was sort of the obligatory ‘in case of funeral’ [suit],” Staton pipes in.

Quips Sommer, a married father of two: “For me, it’s like I’ve worn these three looks that I have 900 times. If you look at pictures of me at parties over the last six years, it looks like I went to three parties and just got different haircuts at the party.”

Speaking of parties, it sounds like the Mad Men gang knows how to have a good time. “It’s a really funny group, like the people on this show are the funniest,” Feldman tells Us. “And I’ve been on sitcoms before where it’s not funny like this show.”

For more from the gentlemen of Sterling Cooper Draper Pryce — including what Feldman’s fiancee thinks of his style, and why all the guys (plus Jon Hamm) took their pants off during a scene with Elisabeth Moss — watch the video now!