Inayat Bunglawala thinks I talk "pure twaddle". Why? Because I suggested in an article about the firebombers who last week attacked the offices of the publishers Gibson Square that part of the problem is that too many liberals have come to accept that it is "morally unacceptable to give offence".

Not so, said Inayat. "If anyone has given ground in this debate," he suggested, "it is surely those who once believed in banning books because they regarded them as being 'offensive'."

Inayat himself has certainly given ground. Once a campaigner against The Satanic Verses, he now believes that no one has the right not to be offended. Inayat's welcome change of heart should not, however, blind him (or us) to the fact that much of the rest of the world has been marching in the opposite direction.

Gibson Square was attacked because it is about to publish The Jewel of Medina, Sherry Jones' novel about the Prophet Muhammad's wife Aisha. It had originally been bought by Random House for $100,000. But the American publishers dropped the book for fear that it "might be offensive to some in the Muslim community". Every other major publisher felt the same. It fell to small independents, with a record of defending free speech, such as Beaufort in America and Gibson Square in Britain, to pick up the pieces.

This is not, however, just a story about a single book or a single publisher. Many within the political and cultural elite have come to think like the directors of Random House. "You would think twice, if you were honest," said Ramin Gray, associate director at London's Royal Court Theatre, when asked last year by the journalist Peter Whittle whether he would put on a play critical of Islam:

You'd have to take the play on its individual merits, but given the time we're in, it's very hard, because you'd worry that if you cause offence then the whole enterprise would become buried in a sea of controversy. It does make you tread carefully.

In June 2007, the theatre cancelled a new adaptation of Aristophanes' Lysistrata, set in Muslim heaven, for fear of causing offence. Another London theatre, the Barbican, carved chunks out of its production of Tamburlaine the Great for the same reason, while Berlin's Deutsche Oper cancelled a production of Mozart's Idomeneo in 2006 because of its depiction of Muhammed. That same year, London's Whitechapel Art Gallery removed life-size nude dolls by surrealist artist Hans Bellmer from a 2006 exhibit just before its opening, ostensibly because of "space constraints", though the true reason appeared to be fear that the nudity might offend local Muslims. Tim Marlow of London's White Cube gallery suggested that such self-censorship was now common, though "very few people have explicitly admitted" it.

There is a similar climate within TV. Just this month, the comedian Katy Brand, of ITV's Big Ass Show, was forced to drop a sketch called "The Imam of Dibley" after lawyers deemed it "culturally insensitive".

Three years ago, after the Birmingham Repertory Theatre had cancelled a production of the play Bezhti, because of protests by Sikhs that it was offensive, Ian Jack, the then editor of the literary magazine Granta, argued for the necessity of self-censorship in a plural society. Whatever may appear to be right in principle, he suggested, in practice one must appease religious and cultural sensibilities because such sensibilities are so deeply felt. The avoidance of cultural pain was more important than the right to freedom of expression.

"The state has no law forbidding a pictorial representation of the prophet", Jack wrote, "but I never expect to see such a picture." An individual might have the abstract right to depict Muhammed, but the price of such freedom was too high when compared to the "immeasurable insult" that the exercise of such a right could cause – even though "we, the faithless, don't understand the offence."

Jack wrote that a year before the Danish cartoon controversy erupted. The response to that controversy showed how deeply embedded such arguments had become. "I understand your concerns," Louise Arbour, the UN high commissioner for human rights, told delegates at the organisation of the Islamic conference summit in Mecca, "and would like to emphasise that I regret any statement or act that could express a lack of respect for the religion of others." The European Union expressed "regret" about the publication of the cartoons. Former US president Bill Clinton condemned "these totally outrageous cartoons against Islam". The then Russian president, Vladimir Putin, not generally recognised as a supporter of Islam, suggested that the Danish government was using the excuse of freedom of expression to protect those who had insulted Islam. The British foreign secretary Jack Straw praised the British media for not publishing the cartoons and condemned as "disrespectful" the decision of some European newspapers who reprinted them.

What we are talking about here is not a system of formal censorship, under which the state bans works deemed offensive. Rather, what has developed is a culture of self-censorship in which the giving of offence has come to be seen as morally unacceptable. In the 20 years since the publication of The Satanic Verses the fatwa has effectively become internalised.

In a plural society, so the argument for self-censorship runs, social justice requires not just that individuals are treated as political equals, but also that their cultural beliefs are given equal recognition and respect. As the British sociologist Tariq Modood has put it:

If people are to occupy the same political space without conflict, they mutually have to limit the extent to which they subject each others' fundamental beliefs to criticism.

I disagree. In a plural society it is both inevitable and important that people offend others. Inevitable, because where different beliefs are deeply held, clashes are unavoidable. And we should deal with those clashes in the open rather than suppress them. Important because any kind of social progress requires one to offend some deeply held sensibilities. "If liberty means anything," as George Orwell once put it, "it means the right to tell people what they do not want to hear." If we want the pleasures of pluralism, we have to put up with the pain of being offended.

I am pleased that Inayat Bunglawala has come to agree with Orwell. It's just a pity that he is still in denial about the harm caused by the broader culture of self-censorship.

This article was amended at 13:15 on October 1 2008