Over the past several weeks, free speech has gotten costlier—at least in France and Israel.

In France, Dieudonne M’Bala M’Bala, an anti-Semitic stand-up comic infamous for popularizing the quenelle, an inverted Nazi salute, was banned from performing in two cities. M’Bala M’Bala has been repeatedly fined for hate speech, and this was not the first time his act was perceived as a threat to public order.

Meanwhile, Israel’s parliament is soon to pass a bill outlawing the word Nazi for non-educational purposes. Indeed, any slur against another that invokes the Third Reich could land the speaker in jail for six months with a fine of $29,000. The Israelis are concerned about both the rise of anti-Semitism globally, and the trivialization of the Holocaust—even locally.

To Americans, these actions in France and Israel seem positively undemocratic. The First Amendment would never prohibit the quenelle, regardless of its symbolic meaning. And any lover of “Seinfeld” would regard banning the “Soup Nazi” episode as scandalously un-American. After all, in 1977 a federal court upheld the right of neo-Nazis to goose-step right through the town of Skokie, Illinois, which had a disproportionately large number of Holocaust survivors as residents. And more recently, the Supreme Court upheld the right of a church group opposed to gays serving in the military to picket the funeral of a dead marine with signs that read, “God Hates Fags.”

While what is happening in France and Israel is wholly foreign to Americans, perhaps it’s time to consider whether these and other countries may be right. Perhaps America’s fixation on free speech has gone too far.

Actually, the United States is an outlier among democracies in granting such generous free speech guarantees. Six European countries, along with Brazil, prohibit the use of Nazi symbols and flags. Many more countries have outlawed Holocaust denial. Indeed, even encouraging racial discrimination in France is a crime. In pluralistic nations like these with clashing cultures and historical tragedies not shared by all, mutual respect and civility helps keep the peace and avoids unnecessary mental trauma.

Yet, even in the United States, free speech is not unlimited. Certain proscribed categories have always existed—libel, slander and defamation, obscenity, “fighting words,” and the “incitement of imminent lawlessness”—where the First Amendment does not protect the speaker, where the right to speak is curtailed for reasons of general welfare and public safety. There is no freedom to shout “fire” in a crowded theater. Hate crime statutes exist in many jurisdictions where bias-motivated crimes are given more severe penalties. In 2003, the Supreme Court held that speech intended to intimidate, such as cross burning, might not receive First Amendment protection.

Yet, the confusion is that in placing limits on speech we privilege physical over emotional harm. Indeed, we have an entire legal system, and an attitude toward speech, that takes its cue from a nursery rhyme: “Stick and stones can break my bones but names can never hurt me.”

All of us know, however, and despite what we tell our children, names do, indeed, hurt. And recent studies in universities such as Purdue, UCLA, Michigan, Toronto, Arizona, Maryland, and Macquarie University in New South Wales, show, among other things, through brain scans and controlled studies with participants who were subjected to both physical and emotional pain, that emotional harm is equal in intensity to that experienced by the body, and is even more long-lasting and traumatic. Physical pain subsides; emotional pain, when recalled, is relived.

Pain has a shared circuitry in the human brain, and it makes no distinction between being hit in the face and losing face (or having a broken heart) as a result of bereavement, betrayal, social exclusion and grave insult. Emotional distress can, in fact, make the body sick. Indeed, research has shown that pain relief medication can work equally well for both physical and emotional injury.

We impose speed limits on driving and regulate food and drugs because we know that the costs of not doing so can lead to accidents and harm. Why should speech be exempt from public welfare concerns when its social costs can be even more injurious?

In the marketplace of ideas, there is a difference between trying to persuade and trying to injure. One can object to gays in the military without ruining the one moment a father has to bury his son; neo-Nazis can long for the Third Reich without re-traumatizing Hitler’s victims; one can oppose Affirmative Action without burning a cross on an African-American’s lawn.

Of course, everything is a matter of degree. Juries are faced with similar ambiguities when it comes to physical injury. No one knows for certain whether the plaintiff wearing a neck brace can’t actually run the New York Marathon. We tolerate the fake slip and fall, but we feel absolutely helpless in evaluating whether words and gestures intended to harm actually do cause harm. Jurors are as capable of working through these uncertainties in the area of emotional harms as they are in the realm of physical injury.

Free speech should not stand in the way of common decency. No right should be so freely and recklessly exercised that it becomes an impediment to civil society, making it so that others are made to feel less free, their private space and peace invaded, their sensitivities cruelly trampled upon.