We’re not overly given to schadenfreude here at Flavorwire, and even if we were, there’s not that much to celebrate in the ongoing downfall of Milo Yiannopoulos. He’s a sad little man who, judging by the comments on pedophilia that have finally brought him down, has years of therapy ahead of him. He’s certainly worthy of contempt, because everything he’s done over the past couple of years has been a cynical, calculated ploy for attention: before he managed to become an icon for the equally sad kiddies of Gamergate, he ridiculed gamers, and before he reinvented himself as a right-wing annoyance, he wrote earnestly about the need for civility on the internet.

There’s a special circle of hell reserved for people like Yiannopoulos, people who know their “cause” and their rhetoric are nonsense, and know that both of those things hurt real people who have done nothing to them, but choose to proceed anyway because doing so brings them fame and riches. But pandering to the right is a dangerous game, because your new friends have as much concern for you as they do anyone else, which is to say very little. If you become a liability, they’ll throw you straight under the bus without a second thought.

So it has proven for Yiannopoulos, who has been kicked to the kerb by his new conservative friends just as quickly as they embraced him. (If you missed the news over the weekend, a video emerged of him apparently condoning relationships between older men and young boys, and as a result, his book has been cancelled by Simon & Schuster, CPac has rescinded its invitation for him to speak, and most hilariously of all, his colleagues at Breitbart are calling for his resignation.)

Good. The world is in enough trouble as it is without cynical toads like Yiannopoulos building careers off its woes. No-one will miss him or his stupid book. But the real question is: why did this take so long?

Alt-right types — along with children on Twitter who don’t have any particular ideology but love Milo because he says naughty things that upset their mothers — have spent much of the past two years defending Yiannopoulos’s schtick on the basis of, yes, freedom of speech. It’s long been a point of bipartisan consensus that freedom of speech isn’t absolute — the classic example is shouting “fire” in a crowded theatre, and subsequent cases have established further scenarios in which First Amendment protections do not apply. These include fighting words and incitement to violence. (They also include false statements of fact; Milo’s employers would do well to take note.)

Let’s be clear: while some of the things Yiannopoulos has said and/or enabled have been thoroughly obnoxious, it’s doubtful that any of them would be examples of speech not covered by the First Amendment. But then, neither would his statements on pedophilia. Until now, Milo’s fan club have cited the First Amendment and painted him as a martyr for free speech every time he’s been slapped down, no matter how tokenistic and overdue those actions have been. Nothing, it appeared, would cross the line, perhaps because there wasn’t one.

Except, whoops, nope, it turns out there is a line! Over the weekend, we found out where that line is drawn: pedophilia! Racism? Totally fine. Sexism? All good. Transphobia? No problem. But endorsing grown men having sex with underage boys? Nope. That, apparently, is where conservatives (and book publishers of questionable morality) draw the line. Pedophilia is such a toxic subject that any hint of association with it is too much.

There’s a whole separate essay to be written about the question of why this is where the line is drawn — there’s something peculiarly American about the combination of pathological obsession with children and complete disregard for any other human beings — and I’m sure psychologists could write entire textbooks about Yiannopoulos and, indeed, about his supporters.

But the more relevant point, at least in the context of this piece, is that free speech has limits. Of course it does. This might seem like an obvious point, and to some extent it’s exactly that. But in a country where sanity and common sense are in increasingly short supply, re-stating the obvious is important and necessary.

With that fact (re-)established, then, it’s instructive to look again at what Yiannopoulos’s supporters have been happy to tolerate. If their commitment to free speech is not, in fact, absolute, then they have made a choice as to what speech is acceptable and what is not. This means that a refusal to censure Milo over his enthusiastic racism, sexism, etc, reflects a thought that those statements are acceptable. This is a case in which a refusal to condemn is essentially an endorsement.

Here’s an example of exactly that: apparently there’s a bunch of Breitbart employees calling for Yiannopoulos to be fired from the site because of what he said in the video over the weekend. I mean, do these people read their own fucking site? Assuming that, yes, of course they do, then they’re presumably perfectly OK with everything that Milo has written in his column on Breitbart, including such gems as “Why Cyberbullying Isn’t Real,” “Trannies Are Gay,” “The Solution To Online ‘Harassment’ Is Simple: Women Should Log Off,” “There’s No Hiring Bias Against Women In Tech, They Just Suck At Interviews,” “Fat People Should Absolutely Hate Themselves,” and on, and on, and on. There are two common threads in these pieces: first, they are the bleatings of a man who desperately wants to be looked at, and second, the man in question’s strategy for getting attention is to shit all over groups of people less fortunate than he is.

None of these pieces, of course, fall foul of the First Amendment, and I’m not suggesting that they do. I am suggesting, however, that when you finally decide that your commitment to allowing this dickhead to say anything he wants isn’t absolute, then you need to look very carefully at what you are fine with him saying. The fact that it’s taken this long for Yiannopoulos’s conservative friends to turn on him says just as much about them as it does about him, and the truth that this sorry episode has revealed — that conservatives care more about hypothetical children than they do about flesh-and-blood women, people of color, trans people, immigrants, refugees and pretty much anyone else who isn’t an unpleasant white man — will remain relevant long after Milo has been consigned to the shitheap of history, his 15 minutes well and truly over.