No one has tested my commitment to liberalism so sorely as Edinburgh University’s Feminist Society. I know I should believe in freedom of speech and changing minds with arguments, not punishments, and all the rest of it. And, trust me, I do. Or rather I did, until the moment Edinburgh’s feminist students said they wanted to kick the Socialist Workers party out of their campus.

The BNP of the left has had a malign influence on public life far beyond its numbers. In the universities, it has been at the forefront of thuggish demands that there must be “no platform” for fascists or supporters of Israel or, it seems, anyone else it disagrees with. The desire to censor has reached the absurd state where the academic left has banned women’s rights campaigners, who have upset transsexuals, and admirers of Friedrich Nietzsche, who have upset students who had not read him but know he was a bad person.

After this disgraceful record, it is worth enjoying the plight of the SWP for hours – maybe weeks. The censor faces censorship. The fanatics who have screamed down so many others could be screamed down themselves.

No one can deny that Edinburgh’s women have good reason to go after the Trots. Like priests in the Catholic church and celebrities in light entertainment, the leaders of a Marxist-Leninist party are men at the top of a hierarchy that demands obedience. Last year, a succession of women alleged that senior figures in the party had demanded their sexual compliance. Rather than tell them to take their cases to the hated “capitalist” courts, the SWP set up its own tribunals. The alleged victims said it subjected them to leering questions worthy of the most misogynist judge about their sex lives and alcohol consumption, then duly “acquitted” the “accused”.

Eleanor Brayne-Whyatt of the Edinburgh Feminist Society has a point when she says that universities will show they do not tolerate “rape apologism and victim blaming” if they order the SWP to leave.

Even if you want to differ, you may find the task of contradicting her beyond you. We have reached a state where arguing that a speaker has the right to free speech is the same as agreeing with his or her arguments. If you say that racist or sexist views should not be banned, you are a racist or rape apologist yourself. Your opponents then go further and accuse you of ignoring the “offence” and “pain” of the victims of racism and sexism have suffered and turn you into an abuser as well.

With remarkable speed this double bind knots itself around its targets. Defend a repellent man’s right to speak and you become that repellent man and his victims, real or imagined, become your victims too. Small wonder so many keep quiet when they should speak up.

Observer readers may not care, as most modern prohibitions on speech are – to put it crudely – instances of leftwing censorship of prejudiced views. If so, you should notice how easy the right finds it to march in step alongside you. Chris Grayling, a Tory bully boy, announced last week that he would quadruple the maximum jail sentence for internet trolls who spread “venom” on social media or, rather, he fed an old story from March to a naive and punitive media.

Even though internet trolls are among the worst specimens the human race can offer up for inspection, there are many reasons not to nod through yet another hardline restriction of personal freedom. Interest groups like nothing better than exploiting the law. We’ve already seen supporters of the McCanns, who were understandably aggrieved by the abuse the family received online, turn into troll catchers. They collected a dossier and passed it to Sky News and the police. The hunters unmasked one of the McCanns’ tormentors as Brenda Leyland, who took her own life within hours of her exposure, a reminder that many trolls are mentally ill and need treatment rather than prison.

Meanwhile, as the free speech campaigners at English Pen reminded me, the white right and far right have learned from the left and can be as politically correct. Their most recent success was to demand that the police prosecute one Azhar Ahmed from Dewsbury. He admitted posting a Facebook message two days after the killing of six British servicemen in Afghanistan: “All soldiers should die and go to hell,” it read.

A disgusting statement, no doubt, but put in different terms, the belief that British troops should not be in “Muslim lands” is a political sentiment, not a criminal act. The court nevertheless found him guilty of the criminal offence of making a “grossly offensive communication”. The prosecutors did not say that he was inciting violence against British troops, simply that he was offensive.

Two can play at that game. The Islamist religious right can respond in kind and demand prosecutions for Islamophobia, and before you know it we will be off on a cycle of competitive grievance.

Only last week, the authorities recalled Tommy Robinson, the former leader of the extreme right English Defence League, to prison – apparently for tweeting that he planned to criticise the police. I carry no brief for the man, but his detention feels all wrong.

It would be far better if social media sites and newspapers stopped inciting people’s ugliest instincts by allowing them to post anonymously. It would be better still if politicians reformed a law that is alarmingly vague.

The state can charge citizens for words that are “grossly offensive,” as Azhar Ahmed found. No government should be allowed to get away with such a catch-all charge. Every sentiment beyond the blandest notions “offends” someone. “Offensive” is a subjective term, which is wide open to political manipulation by loud and vociferous interest groups and the government of the day.

The only respectable reason for banning organisations or punishing individuals is if they incite violence against others. Unless feminists can prove that the SWP promotes rape as a matter of party policy – and I don’t think they can – they remain free to despise it, harangue it and oppose and expose its many stinking hypocrisies, but they have no moral right to order it off campuses.

I know I am going to regret writing that last sentence. Indeed, I am regretting it already. But it remains the case that a country where it’s a crime to be offensive is a country where everyone can try to ban everyone else.