You can't get much more egregiously wrong and wicked than the views expressed by al-Muhajiroun. But they are currently operating in what I hope will be a very brief legal air-pocket. As with the BNP, while they are legal and are being given a platform by independent organisations, they cannot go unchallenged.

Some months ago I received an invitation to debate sharia law from something called the "Global Issues Society" (GIS). They claimed to be a student society. I accepted the invitation in good faith and was assured – repeatedly – that the platform and chairperson were neutral (the chair, as I later discovered, most certainly wasn't). My office was in regular communication with the GIS stipulating the conditions under which a safe debate could be had opposite an extremist like Choudary. We were sceptical of GIS from the start. We strongly suspected that they were some sort of front group. But we couldn't satisfactorily confirm the fact. Despite our concerns, and our knowledge that it was a possible ambush, I gave them the benefit of the doubt and decided to attend the event in good faith.

It became clear as soon as people arrived that it was an ambush. We had been assured by GIS that security guards would be hired, but there was a problem – they were members of al-Muhajiroun. Their only purpose was to ensure the men and women remain segregated – a condition never stipulated to us by GIS and a gender-apartheid which I would never accept at any event I was speaking at. Before I even arrived a member of the public was assaulted and a mini-riot ensued. I was advised to keep away from the hall until the arriving police had got a hold on the situation.

Of course I have no "supporters" or "followers" who attend my talks. I don't have a "faction". I am just one voice who people can either agree with or – more commonly in these pages at any rate – disagree with.

The easy thing to do is to turn down invitation to debates like these. I speak at various events, including student societies, several times a week. I can't think of the last time I spoke at an event where a majority of people agreed with me. But the point of free-speech isn't continuous and tedious hymn-singing. Audiences I speak to are often openly hostile and I know my arguments might fall on deaf ears with 99% of the audience. Yesterday I was surrounded on the street by 100 thugs yelling "fucking kuffar" and "coward". There was some pretty pathetic homophobic abuse too. The easy thing to do is to ignore these people. But I don't think we can.

I have had too many experiences of finding just one person or a couple of people, usually young, who have genuinely never had their minds opened up by contrary opinions. Even when they don't agree with a new opinion, they can at least become aware that the point of free societies is that we don't agree. On anything. And certainly – for well known reasons – never unanimously. Never finally. The debate is the thing.

At events I regularly meet young Muslims and non-Muslims who have simply never heard arguments put for why liberal democracy is, though not perfect, our only achievable, messy, hope. I think it important that they hear someone speaking in defence of religious pluralism, women's rights, gay rights, and actual diversity in society. I also think it important that they hear religions critiqued, satirised and treated in the same robust manner in which the rest of us have our most cherished ideas treated.

Past experience across the country has shown it is even possible to chip away at the mindset of radicalised Muslims – sowing seeds of doubt. Even if it is just one member of the audience who is receptive to the anti-totalitarian possibility it is vital to do this. It is the reason why I debate.

Some people will say that al-Muhajiroun should not be given the oxygen of publicity. Yesterday showed why bringing them out in the open and challenging their ideas is necessary. It reminds the government, the press and British citizens of the true nature of these fundamentalist thugs who are not just going to disappear. Not since Oswald Mosley's British Union of Fascists (BUF) have we seen intimidation like this on the streets of London. Like the BUF, they will resort to violence the moment their fascist views are challenged. Every piece of publicity al-Muhajiroun gain is additional ammunition to encourage the government to shut down a group whose members have regularly exceeded our incitement laws and even involved themselves in acts of terror.

Centre for Social Cohesion research (pdf) shows that one in seven Islamist-related terrorist convictions are related to al-Muhajiroun. I asked Anjem Choudary about this on the pavement yesterday. He said that I was a liar. He said that the police were liars, that the courts were liars and that the whole system of the state was a liar. And when I quizzed him about his convicted terrorist friends? "Muslims are always innocent" he said. And there you have it. I thought he thought that. I thought he wouldn't say that. But he did. As I was reminded in various ways the other night, it's always good to know exactly what kind of an extremist it is that you're dealing with.