Of late, a new variation of the old chestnut "I'm not racist but …" has emerged. It goes: "I've got nothing against Muslims, it's Islam I hate". Otherwise known as the "Islam is not a race" argument.

After I wrote about Richard Dawkins's snide attack on the supposed dearth of Muslim scientific and cultural achievement, some critics hit back along these lines. It is acceptable to criticise and belittle Islam because it is a religion, not an ethnic grouping – and therefore fair game.

Technically, they are right – Islam is not a race. But too often, those who deploy the argument, are borrowing from the Bill Clinton school of sophistry: "I did not have racist relations with that religion".

Dawkins himself dedicated a large part of his riposte to a dissection of whether race is a biological or social construct. The argument over Islam and race was a "simple semantic disagreement", he said, before proceeding to define race according to the dictionary.

So what does the dictionary say? Racism, according to the Oxford English Dictionary, is "prejudice, discrimination, or antagonism directed against someone of a different race based on the belief that one's own race is superior". But what does this mean in practice?

Under British law, Jewish people are classified as belonging to a race (something that Dawkins, incidentally, disagrees with) since they are deemed to have a shared culture and history that goes beyond the religious sphere. Do I share history, culture and other reference points with Muslims around the world that go beyond the practice of Islam? Definitely. But it is a loose, secular feeling.

Does this make me immune to discrimination that Muslims face? Certainly not, given the long hours I have spent in immigration queues, undergoing extended background checks and visa processing times. So clearly, in that sense, Islam is not a race, but Muslims are. It's entirely legitimate to question and interrogate Islam as a religion. It is not fair to do so against Muslims based on their religious or cultural identity.

Equally, it is disingenuous to claim that Islam has no colour. There is actually quite a strong racial dimension to Islamophobia. Muslims in the UK are predominantly brown, Asian or Arab, and there have been instances where non-Muslims from Asian communities have been lumped together with Muslims and discriminated against.

After the 9/11 attacks, some bearded and turbaned Sikh men found themselves coming under hostile scrutiny. Following the more recent Boston bombings, some media outlets described suspects as being of "Muslim appearance" – whatever that is. In the wake of Theo Van Gogh murder, racist targeting of Muslim immigrants increased.

When discrimination against some eastern Europeans in the UK is called racism, you don't hear cries of "Polish is not a race" to justify plain prejudice. The fixation on terminology and not the reality suggests a society that does not want to come to terms with the creeping ugliness of hatred. The likes of the BNP and EDL lack even a basic grasp of the rudiments of Islam, let alone an ability to parse religion and race.

Racism is behaviour, not an informed academic position. I doubt that anyone abusing Muslims in the street, or defacing a mosque, or snatching a veil off a woman's face, has paused to examine their premise beforehand. The argument that Islam is not a race is a cop out. It's time that we dispensed with it once and for all, because it prevents us from identifying acts motivated by hatred for what they really are. Islam might not be a race, but using that as a fig leaf for your unthinking prejudice is almost certainly racist.