The racist goons in Charlottesville have inspired a fresh debate over whether the government should allow speech by racists, goons, and assorted other troglodytes.

For some, the answer is clearly no. "The ACLU Needs to Rethink Free Speech," argues a fellow with the UCLA School of Law. "Censor White Supremacy," advocates a writer in The Week. "Speech in America is Fast, Cheap, and Out of Control," bemoans campaign-finance scourge Richard Hasen in the Los Angeles Times.

Here in Virginia, Gov. Terry McAuliffe (D) has temporarily banned demonstrations at the Lee monument in Richmond. City leaders in Portsmouth are debating whether to adopt a similar ban, at least with regard to hate groups. On the other side of the country, House Minority Leader Nancy Pelosi (D-Calif.) wants a permit revoked for an alt-right rally. Transit authorities in New York and Washington have been trying to limit controversial (and even noncontroversial) advertising. And so on.

Those demanding censorship sometimes try to put a fig leaf on the demand. "This executive order has nothing to do with infringing upon First Amendment rights," McAuliffe claimed, even though that was its whole point. In The Week, Matthew Walther contends that defending "abstract rights" of even the worst people is not "to defend speech but to demean it, to diminish it to the level of undifferentiated random noise." Sure, that's probably it.

Defenders of free speech have offered some familiar but still trenchant rebuttals:

Who decides what is acceptable? Do liberals really want to give a Republican Congress and, for Pete's sake, Donald Trump the authority to decide which speech to punish?

Where do you draw the line? In Europe, courts have fined and imprisoned people not just for classic hate speech, but also for "glorifying terrorism" with a puppet show, saying mean things on Facebook, and posting "cruel humor" on Twitter, among a great deal else.

What about blowback? As Washington Post ombudsman Margaret Sullivan suggested in a recent column: "Imagine a civil rights march that is shut down because officials fear a violent response from racists." She quotes Justin Silverman, of the New England First Amendment Coalition, who points out that until relatively recently "rallies for equality and civil rights were considered offensive and unpopular."

These are all good arguments, but they suffer from a common flaw: They are conditional. They allow for the possibility that censorship might be acceptable if we could ensure that the right people imposed it, that they would draw the lines in the right place, and that good ideas would never be censored.

In the real world, those conditions cannot be met all the time, which makes the arguments powerful. But there are two other points that need drawing out as well.

The first is that the right to speech is not merely instrumental. In The Week, Walther argues that "freedom of speech is not a first-order good; it exists only to facilitate the flourishing of… society." Ergo, speech that doesn't facilitate somebody's conception of flourishing can be censored. (You hear the same arguments in the campaign-finance context: Speech by powerful interests can be curtailed in the name of a "level playing field.")

This is just drastically wrong. It treats people as mere means to somebody else's end, as Kant would say—not as ends in themselves. The blood-soaked history of the 20th century should have discredited that collectivist vision of society for all time, but apparently it still holds appeal in some quarters.

The other point that needs drawing out has less to do with the people who are speaking than the people who stand in judgment of the speakers. The current debate has focused on whether certain vile ideas have any value, and whether the people who espouse them deserve to be able to voice them. This looks at the question from the wrong end.

The more important question is: What right does anybody have to shut them up? That is the real threshold issue.

By way of analogy: Smith might not care for country music. He might, in fact, despise it. But Garcia is still free to play country music all he wants. And here is the crucial part: Garcia is not free to do so because country music is inherently deserving of protection. Or because, if country music is banned, classical music might be next. Or because someday Garcia might be in charge and want to stop Smith from playing the Beatles.

No. Fundamentally, Garcia is free to play his country music for one immutable reason: Smith has no right to make him stop. What comes out of Garcia's stereo is simply not for Smith to say.

And the same goes for Garcia's mouth.

This column originally appeared in the Richmond Times-Dispatch.