Hugh Hefner died at age 91 on Wednesday, and with his passing goes one of the last vestiges of the much-mythologized sexual freedom of the ‘60s and ’70s. As the founder of Playboy Magazine and its sprawling, iconic brand, he was a trailblazer of sexual liberation. He was a patron of jazz and a proponent of civil rights, making high-minded writing and nude centerfolds part of the thinking man’s cultural repertoire in ways that can still be seen in the American mainstream today.

But in the midst of appreciating his life, it’s important to remember — and contextualize — who and what that sexual revolution served. Hefner’s version of liberating himself from the prudish mores of his youth directly led to collecting young blonde women around him as if they were posable Barbies. A self-described late-bloomer who didn’t even masturbate until the age of 18, Hefner fought against his own repression to make himself into a man who could get whatever he wanted. Hefner himself claimed to respect women, and one can only hope that he did — that his dozens of Bunnies were all consensually and enthusiastically with him, in a orgiastic Viagra-fueled polyamorous arrangement that is captivating precisely because of how unlikely it is.

But what he wanted was women who existed almost entirely within his erotic gaze. It is no surprise that Hefner’s girlfriends were nearly identical. The New York Times Magazine described them as “most of them blond, many with names ending in a vowel and all of them with mammary tissue that appears to have been injected with helium.” By his own admission, in the same article, he admits to imposing a 9 p.m. curfew on his seven girlfriends “to keep them from dating anyone else.”

Over the past few years, the sexual escapades of stars in the ‘70s have time and again exposed seriously worrying, if not outright abusive, patterns of behavior enacted upon young and impressionable women. To Hefner’s credit (the era has a low bar for merit), he is widely regarded as a man of unimpeachable character, and that reputation stayed intact through a dead Playmate and a few tell-all books. On the other hand, Hefner’s Playboy was revolutionary because of how much it permitted men to fantasize about; but what Hefner and Playboy were unwilling or unequipped to address was the fallout — that those fantasies often came at the cost of women’s bodies.

The most damning moment in Hefner’s history is not even about Hefner, exactly; It is about Bill Cosby. In 2015, model Chloe Goins charged Cosby with assault at Hefner’s storied Playboy mansion. Goins alleged that Hefner introduced her to Cosby, and that Cosby then slipped a narcotic in her drink. Cosby’s drug of choice was the quaalude — that relic of the ‘70s used by Roman Polanski, Andy Warhol, David Bowie, and, yep, Hefner himself. Holly Madison, one of the few girlfriends (“Playmates”) who didn’t sign a nondisclosure agreement, wrote in a controversial and difficult-to-verify book that Hefner called quaaludes “thigh-openers.” And though there isn’t a shred of evidence, anywhere, suggesting Hefner used those drugs for assault or abuse, Madison details a culture of appearance-based objectification and control that standardized the look of his girlfriends. When fellow girlfriend Kendra Wilkinson-Baskett attacked Madison for her book, she did so by rejoindering that Madison was just embarrassed that she was the dirtiest slut at the Playboy Mansion.

Oh, the mansion — which Hefner sold in 2016, but resided in until his death — is almost more storied than Hefner himself, an erotic fantasyland where the women are blonde and willing and the men smoke cigars and play games. The mansion where, Madison alleges, every woman who walked through the door was photographed and then ranked. The mansion, where Cosby allegedly assaulted Chloe Goins. In 2015, Goins charged Hefner in her suit too, arguing that “Defendent HEFNER knew or should have known” that Cosby would endanger Goins. She added: “Defendant HEFNER’s actions of inviting and hosting young and impressionable and possibly minor children to his residence, and providing alcoholic beverages and or foreign substances was negligent at the very least.”

It’s a telling accusation, and one that gets to the heart of Hefner’s legacy. Hefner himself may have been an upstanding man of personal integrity; indeed, in his desire to dismantle social mores, he broke many barriers. But he was the king of the male id — the patron saint of transgressing social mores, who reveled the nude models that surrounded him. Could he really have been unaware of how that might go wrong? Playboy was a magazine for men that featured primarily women — naked or near-naked women, including famously Marilyn Monroe — posed in centerfolds like they were served up in platters for voracious men. There’s a place for that, to be sure. But Hefner’s world — a world in which, by the end, he exclusively wore pajamas and a smoking jacket in — reduced everything and everyone to a sexual gambit played on his terms. Hefner reduced women to a pair of tits and a cotton tail on the ass, because that’s what he liked. Perhaps he brilliantly exposed desire — but whose desire, and at what cost?