Monday night's Channel Four studio debate, Muslims and Free Speech, exemplified the double-standards of some Muslim leaders. They object to people offending their religious sensibilities, but happily cause offence to others whenever it suits them.

Taji Mustafa, of the rightwing Islamist group Hizb ut-Tahrir (HuT), failed to make any differentiation between criticisms and insults. He seemed to suggest that any criticism of Islam is an insult and that all such insults are unacceptable: "What Muslims will not accept ... [is] gratuitous insults about their beliefs, their faith or anything dear to them," he said. Such sweeping exclusions appear to leave little room for a genuinely free exchange of ideas.

HuT's intolerance is not surprising, but its hypocrisy in breathtaking. While Mustafa rejects criticisms of Islam that he finds insulting, his organisation has a long history of deliberately insulting Jews: "Jews are a people of slander ...a treacherous people," according to one HuT leaflet.

Moreover, Mustafa's organisation used to openly call for the killing of gay people and its constitution is an agenda for clerical fascism. But this makes bad PR, so HuT recently removed the constitution from its website. I archived a copy before it was deleted. I can see why they wanted it removed from public view. It is jam-packed with opinions that most people would find deeply offensive. But they don't care about causing offence because they believe that it is their god-given right to offend non-believers.

The HuT constitution calls for the creation of a theocratic dictatorship, where non-Islamic political parties are banned and where the only law is "divine law." It stipulates the execution of Muslims who turn away from their faith. It demands that women must obey their husbands and that women should be debarred from ruling positions in society.

Such views are an insult to women. But HuT would scream blue murder if anyone dared demand that its right to be sexist should be restricted in any way.

HuT is, of course, on the radical fringe of Muslim opinion, and cannot be taken as representative of Muslims as a whole. So, what about mainstream Muslim attitudes?

Even the supposed Muslim moderates on last night's programme exuded a whiff of hypocrisy. Ibrahim Mogra of the Muslim Council of Britain (MCB) claimed: "We do not wish to impose our way of life on anybody. All we want is to live in respect with one another." Fine sentiments. Shame about the reality.

Where is the MCB's respect for other people when it denounces homosexuals as "repugnant" and "immoral"? Isn't this language wilfully offensive? Moreover, the MCB does want to impose its beliefs on others. It has worked hand-in-glove with Christian fundamentalists to support homophobic discrimination and to resist every gay law reform of the last decade. The MCB fought to maintain discriminatory laws like the unequal age of consent and Section 28, and it opposed civil partnerships and protection for gays against discrimination in the workplace. It has agitated to impose its homophobic policies on the rest of society by attempting to maintain homophobia as the law of the land.

None of this would matter much if the attitudes of Mustafa and Mogra towards free speech were marginal. Unfortunately, this is not so. They appear to be representative of the majority of British Muslim opinion on this subject. According to an NOP poll for Channel 4's Dispatches special, What Muslims Want, which was broadcast in August, two-thirds of Muslims in Britain oppose free speech if it offends their religious beliefs. They want to make it a crime to cause them offence. People who insult Islam should, they say, be arrested and prosecuted.

In other words, they want privileged legal protection against any criticism of their beliefs that they find offensive. Their aim is to secure a legally binding veto over what other people, including other Muslims, are allowed to say about them and their faith. Put simply: in the name of being spared offence, they want to censor other people's opinions. Moreover, they are not demanding protection from offence for everyone - only for Islam and other religions. In effect, they are seeking unique protection for believers and their beliefs.

One big danger is that any restriction on freedom of speech in the name of preventing offence is likely to be exploited by orthodox Muslims to close down debate within their own community - to silence dissenting liberal and progressive Muslim voices who raise uncomfortable issues such as domestic violence, forced marriages, child abuse, honour killings, female genital mutilation and queer-bashing.

I have seen this happen. The suppression of critics within the Muslim community is already excessive. Some Muslims have been accused of insulting Islam because they have a non-traditional interpretation of the Qur'an. These include adherents of minority Muslim sects like the Ahmadiyya. They have experienced intimidating late-night visits from members of supposedly moderate Muslim organisations. Imams who question misogyny and homophobia have been threatened with being stripped of their qualifications and office.

Others have faced threats of expulsion from their mosques for deviating from the Muslim party line. Some have had their children and elderly parents menaced. Muslim reformers, such as Irshad Manji, who advocate a modernist Islam for the 21st century, live in fear of assassination by other Muslims. Indeed, the MCB went out its way to expose Manji as a lesbian in a seedy bid to discredit her ideas. In other words, Muslims who reject free speech have already claimed their first victims and these victims are fellow Muslims.

Some Muslim spokespeople on last night's programme seemed to blur any distinction between insults and violence, as if both were equivalent and interchangeable. I accept that incitements to violence against Muslims (or anyone else) are a step too far. They are against the law and rightly so. Violence and threats of violence are inimical to the free exchange of ideas. All communities have a right to live without fear of violent attack.

Hence the OutRage! campaign against certain Jamaican reggae singers. I did not oppose them because they are homophobic, but because they advocate the murder of lesbians and gay men. I can put up with their homophobic insults, but not with their public incitements to shoot, burn, hang and drown homosexuals. Black gay Jamaicans have a right to live their lives without being bombarded with exhortations to kill them and without the fear that they will be hacked to death by a homophobic mob.

But sections of Muslim opinion (and their Christian and Judaist fundamentalist counterparts) go much further. They want to outlaw the giving of offence.

I would never wish to cause gratuitous offence to anyone, including Muslims. We should all try to act in respectful and kindly ways towards each other, recognising and respecting difference. However, difference sometimes involves views that we find disagreeable and offensive. In a democracy, we have a legitimate right to criticise offensive opinions but not to ban them. Once you start banning views where do you stop? Almost every opinion is offensive to someone. We'd end up with no debate about anything.

So, while not aiming to cause offence, I cannot accept that Muslims, or anyone else, should have a right to censor my opinions. This is not because I regard my views as important or deserving of special attention. It comes, in part, from experience.

I lived through the McCarthy-style witch-hunts in my native Australia in the 1960s. I nearly lost my job because I was labelled a communist for opposing the US and Australian war against Vietnam. I was told my views were offensive and unpatriotic - an insult to Australian soldiers who were fighting and dying in the jungles of Vietnam. I experienced first-hand the threat to personal freedom involved in policing people's thoughts and opinions. Ever since, I have concluded that tolerating views that I might find offensive is an essential aspect of a free and democratic society.

Indeed, I defend absolutely the right of Muslims to express their views, even if they offend me and insult the heart of my being. That's democracy.

In January this year, the then leader of the MCB, Sir Iqbal Sacranie, was questioned by the police after he insulted gay people, by suggesting we are harmful, immoral and diseased. Although his views are odious, because I believe in free speech, including the right to give offence to me, I spoke out against Sir Iqbal being prosecuted. All I am suggesting is that Muslim leaders and organisations extend the same right to me and to others.

For saying these things I will probably be denounced, yet again, as an Islamophobe by members of the MCB, the Muslim Association of Britain and the Islamic Human Rights Commission - and their "leftwing" allies in the Socialist Workers party, Respect and the Stop the War Coalition. I have lost count of the number of times that I have heard them say things like: "Peter Tatchell is a Muslim hater ... he is working for the BNP ... he wants Muslims expelled from Britain," and so on. It is all complete nonsense but they keep repeating it in the hope that the mud will stick.

For the record, I have never attacked Islam per se, or Muslims in general. I have made a specific critique of Muslims who reject (or who want to restrict) free speech.

Those who cry Islamophobe will no doubt accuse me of targeting "weak and vulnerable Muslims". Playing the victim card is, of course, no substitute for rational argument. Besides, my targets are bigots and censors - not Muslims.

Finally, let me conclude with some good news from the Channel 4 NOP poll. Despite the bullying and intimidation of some Muslim leaders and organisations, one-third of British Muslims continue to support freedom of speech - even when it causes them offence. They realise that being able to speak freely is in the interests of Muslims too. It gives them the right to speak out and say critical, challenging things that Bush, Blair and Olmert, for example, may find offensive. It also ensures that there can be an open debate within Muslim communities about different understandings of their faith.