It was "appalling", "vile", "hateful". It was "incredibly offensive". It was "rude, bigoted and downright insulting". In the 24 hours following the publication of Julie Burchill's Observer piece headlined "Transsexuals should cut it out", more than 1,000 emails arrived in my inbox and 2,952 comments were posted online, most of them highly critical of the decision to publish what one correspondent called "her bullying nonsense".

The piece in question was a defence of her friend, the columnist Suzanne Moore, who claims she has been driven off Twitter by a vociferous campaign from transsexual people. Moore had contributed an essay on women's anger to an anthology of polemical writing. Women were angry, she wrote, at the effect of government policy on the weakest members of society, many of whom happened to be women, and they were angry, among other things, at "not having the ideal body shape – that of a Brazilian transsexual".

This, wrote Burchill, led to Moore being "monstered" by a lobby that Burchill said would rather silence Moore than decry the idea "that every broad should look like an oven-ready porn star". She said the lobby was now saying it was Moore's refusal to apologise that "made" them drive her from Twitter, presumably in the name of solidarity. Some of the language was gratuitously offensive; to repeat it here would be to add insult to injury.

The ensuing storm was notable both for its vociferous nature and for its individuality. A controversial issue will often bring a blizzard of identikit protest of apparently confected anger but while clearly this lobby was organised most of the emails and letters we received were personal and heartfelt. And they were not only from trans people. Concerned readers with no connection to the trans lobby felt hurt that a minority that could expect to be protected by a liberal publication was being attacked in an extremely insulting manner.

"Would you have run the article if it had contained similar slurs regarding people of colour or people with disabilities?" was a typical question.

Many correspondents pointed out that our own editorial code states "... we should not casually use words that are likely to offend" and cited clause 12 of the national Editors' Code: "The press must avoid prejudicial or pejorative reference to an individual's race, colour, religion, gender, sexual orientation or to any physical or mental illness or disability." But note, this is a safeguard for individuals; it offers no protection for groups or "communities".

The problem for the editor, and the reason why he took the decision to take the piece off theguardian.com, is that he did not feel he could defend it in that form. It also breached the standards that the paper expects others to uphold when they submit comments to the website. They state: "We will not tolerate racism, sexism, homophobia or other forms of hate-speech or contributions that could be interpreted as such. We recognise the difference between criticising a particular government, organisation, community or belief and attacking people on the basis of their race, religion, sex, gender, sexual orientation, disability or age."

The editor told me: "This clearly fell outside what we might consider reasonable. The piece should not have been published in that form. I don't want the Observer to be conducting debates on those terms or with that language. It was offensive, needlessly. We made a misjudgment and we apologise for that."

He said he had "no doubts whatsoever" about his decision to remove the piece from the site. "It was a mistake to publish it. I could not let the mistake stand. I didn't want that legacy for the Observer. The idea that I would compound a mistake by continuing to publish the piece online is absurd. It follows that if you make a mistake, you try to correct it. I had to do what I could to address this error. Protestations about censorship were irrelevant. It was not a complicated decision to make. The responsibility I had was to try to make amends to a group of people we had needlessly and mistakenly offended. I had no responsibility for, or interest in, the sanctimony of other news organisations. It wasn't about free speech, or Leveson or Lynne Featherstone [the Liberal Democrat MP who called for Burchill's sacking]. It was about a personal desire – and decision – by me to address the offence caused to a group of people. The rest was noise."

He made his apology in the following statement on the website, which appeared last Monday: "The piece was an attempt to explore contentious issues within what had become a highly charged debate. The Observer is a paper that prides itself on ventilating difficult debates and airing challenging views. On this occasion, we got it wrong and, in light of the hurt and offence caused, I apologise and have made the decision to withdraw the piece."

It's worth noting that because the piece appeared on theguardian.com, many readers felt it had emanated from the Guardian. Let's be clear: while both papers share the same digital platform they are edited separately.

A collective failure of editing led to this piece appearing in the form that it did. "We will scrutinise further the manner in which this process needs improving," said the editor.

Anyone commissioning Burchill can expect to receive some high-flown polemic, something her regular readers have come to expect; a Burchill column is rarely uncontroversial. Several senior staffers saw the piece before it appeared and could have urged wider discussion on the impact of the piece. I include myself in this; I saw the piece when it arrived on Saturday morning but hesitated to suggest changes (my role as readers' editor is not to intervene in advance of publication). That's something I now regret.

Philip Pullman once said: "No one has the right to live without being shocked. No one has the right to spend their life without being offended. Nobody has to read this… Nobody has to pick it up. Nobody has to open it. And if they open it and read it, they don't have to like it."

Which, of course, is correct, but freedom of expression means nothing if gratuitous insults mask the very message that is being conveyed. In publishing those insults, the Observer fell below the standards it expects others to uphold. There was no other option but to withdraw the piece and apologise.

So how does the Observer move on from here? The editor says that discussions with representatives from the trans community will take place over the coming weeks. These discussions will be an opportunity to listen and also to debate the issues raised by this incident. A lesson has been learned.

reader@observer.co.uk