Like many people before and after him, the late hockey legend Conn Smythe—the former owner of the Toronto Maple Leafs and builder of Maple Leaf Gardens—hated high school. Smythe enrolled at Upper Canada College in the early 20th century, one of the oldest, most venerable boys’ private schools in the country. According to his biography he detested UCC so much he left after only a year and a half and enrolled in a downtown public school.

But unlike most people who hate their high schools, Smythe insisted that his sons Hugh and Stafford attend his loathed alma mater because he believed, “once a boy had gone to Upper Canada, he would never again be in awe of great family names, money, power or social standing. He would know that although a good private school can produce the best, it can also produce the worst.”

In other words, for Smythe, the image of his sons attending an elite, Old Boy institution—and possibly enjoying themselves—was more tolerable than the image of his sons admiring and revering such a place from a distance. If Smythe had become a father in this day and age, however, it’s unlikely he’d have to worry about his sons’ being unduly impressed with an institution like UCC.

Old Boy Elitism—boys’ private schools, college fraternities and private men’s clubs—may produce CEOs and politicians like clockwork. But their lustre, thanks to an increasingly egalitarian culture (one that values diversity and dislikes douchebaggery), has begun to fade. The stereotypical old boys, i.e. the white, male, snooty upper-cruster, isn’t merely the bane of his favourite punching bag: the feminist, the hippie, or the queer. He is the bane of all kinds of people, real and imagined: Ponyboy in The Outsiders, Ryan Atwood on The O.C., Frank “The Tank” Ricard (a.k.a Will Farrell) in Old School, and most recently, the Jewish staff of Camp Firewood, on Netflix’s revival of Wet Hot American Summer, who are terrorized routinely by a group of sneering sweater set clad guys across the lake. Popular culture, not to mention life (ahem, Donald Trump), is littered with deeply unlikeable Old Boys.

It is hardly surprising, then, that Finance Minister Joe Oliver cancelled a sold-out talk on Wednesday at The Cambridge Club, a private Toronto men’s club he has belonged to for 42 years. The Cambridge Club, for those not yet acquainted with “the downtown playground for Toronto’s most successful executives,” is as Old Boy as they come. It is home to leather armchairs, Scotch tastings, and according a story in the Financial Post this year, “bound volumes of Playboy.” Oliver was scheduled to speak about economics at the club on Wednesday but pulled out last minute when Liberal MP Chrystia Freeland took umbrage with the event’s setting. Her beef: Why should Canada’s finance minister speak at a private men’s club where women aren’t welcome beyond the co-ed dining room? Twitter took the bait and soon Oliver’s name was trending. He had become, in a matter of minutes, one of the most well-known and reviled Old Boys in the country.

Freeland’s impassioned performance at the Cambridge Club on Wednesday wasn’t exactly moving. Having attended Justin Trudeau’s ill-conceived “Ladies Night” event last year, at which no actual food was served (just candy) and the all-female guests were forced to submit questions to Trudeau anonymously into a box a la gym class, it is hard to take the Liberal Party’s feminist protestations seriously. (It is also telling that no such furor erupted in the wake of Olivia Chow’s speaking engagement at the Cambridge Club during last year’s Toronto mayoral campaign.)

But it is interesting how quickly Oliver backed down—almost as though the second his people got a whiff of disapproval from the mostly younger crowd deriding the minister on Twitter and Facebook, they told him to throw in the towel—or in this case, the monogrammed bathrobe. This is a man, remember, whose photograph is hanging on the club’s wall.

Old boys may rule the world, but judging by today’s snarky online theatrics—post after post of dorky photographs lifted directly from the Cambridge Club’s website—it’s clear they are regarded by almost everyone outside their stuffy walls as painfully uncool. (I don’t know about you, but I’d rather my pornography not be bound in leather). Their influence remains enormous, of course. White male privilege—as the kids these days like to call it—is far from dead.

But Oliver’s swift rejection of the Old Boy podium may indicate that it is no longer fashionable for an elected official to formally address an insular group of rich, white, men who subsist on Lagavulin and d–k jokes; rich white men who are so threatened by the supposed scourge of political correctness (in reality, easier to avoid outside a college campus than Rob Ford at a gay pride parade) that they are willing to pay thousands of dollars a year for likeminded friends with whom they can lounge around in expensive towels and make snide remarks about Caitlyn Jenner. Good riddance.