I first started calling myself a “moderate Muslim” in 1989 in the aftermath of the Salman Rushdie affair. Watching bearded men in Bradford burning copies of The Satanic Verses I wanted to signal that I was not like them, that I believed in free speech and tolerance and being reasonable and civil, and that I did not see a contradiction in living within the value structures and laws of this country and being Muslim.

The claim that most Muslims are moderate is rarely challenged, which is why I have spent the last month travelling around the country talking to ordinary Muslims about the word. I anticipated disagreement on what defined moderate; what I did not expect was universal hostility to the very phrase and yet everywhere I went the message was the same: don’t call us moderate.

“I see it as a criticism,” one woman in Luton told me. “You are giving me this label based on how I look and how I dress.” Her male friend said he found the word “offensive”, adding: “Are you saying I’m only 50% Muslim? When someone says to me ‘you’re moderate’ it suggests to me they’re saying ‘you’re not fully Muslim’.”

It was surprising to find a term that I had always assumed was favourable and benign being so roundly condemned

The men and women I met told me they found it infuriating that they could be devout in their practice of their religion but they would only be considered “moderate” Muslims – since moderate was often taken to mean not hugely observant. Meanwhile those who had committed appalling acts of terror – and who were often far from religious in their earlier lives – would immediately be considered “real” or “full” Muslims. It was surprising to find a term that I had always assumed was favourable and benign being so roundly condemned. It may once have been useful but the phrase is no longer fit for purpose.

This is about more than politeness and semantics. The language matters because if ordinary Muslims feel they are being divided into camps, into “good” (moderates) and “bad” (non-moderates) then any statements made by politicians will be heard with great scepticism, especially if for many Muslims “moderate” has come somehow to mean more secular.

The other danger is that if politicians merely assume that most Muslims have views that are moderate they may be blinding themselves to the scale of the challenge they face. A recent BBC survey of 1,000 British Muslims revealed that while 95% said they felt a loyalty to Britain and the vast majority had no sympathy for the perpetrators of the Paris attacks, 27% did have some sympathy for the motivations behind the Paris attacks and 45% said clerics preaching violence against the west could be justified as being in touch with mainstream Muslim opinion.

The men and women I met were uniformly horrified by the atrocities perpetrated by Islamic State and rather insulted when asked if they could shed any light on why anyone from this country would wish to travel to Syria. They disputed the suggestion that their Islam had anything in common with the ideology of the jihadis.

But if by being moderate we mean believing in freedom of expression and an acceptance of equality in gender and sexuality, it is by no means certain that the majority of Muslims are moderate, as the word is commonly used. One young student in Leicester told me that unlike many of his fellow Muslims he would shake a woman’s hand. He still believed in gender segregation at public events but his stance on handshaking meant he defined himself as a moderate.

While everyone I talked to would begin by firmly stating how all Muslims they knew were tolerant and how Islam was about moderation, when I asked how others in their communities in Luton, Bradford and Leicester would react if a loved one told them, for example, that they were gay, or where they stood on the idea of free mixing between the sexes, it quickly became apparent that they held views that many would not consider moderate at all.

In trying to encourage more Muslims towards modernity the mistake is to assume that moderation equates to liberalism. The implication in the term “moderate Muslim” is someone who is not especially devout. But being devout is not antithetical to being moderate. That is why it is so important to make a distinction between being moderate, conservative and religious. Many of the people I talked to considered themselves devout Muslims but told me they believed Islam was intrinsically moderate.

To assume most Muslims are moderate is to imagine that progress is inevitable but if we want the next generation of Muslims to have values we associate with being moderate, we need to be prepared to make an argument for a progressive Islam. That is not possible if politicians employ terms many Muslims find alienating.

The distinction is not between those who are moderate and those who are dangerous extremists. It is between those who continue to insist that Islam can somehow be disentangled from culture, geography and history – on issues such as female dress, gender mixing and homosexuality for example – and those who recognise that Islam has always adapted and incorporated the specific culture of the country where it is practised. These are not lesser Muslims so rather than using “moderate”, let us call those who follow an Islam that is open-minded, liberal and tolerant what they actually are: British Muslims.

• In Search of Moderate Muslims, presented by Sarfraz Manzoor, is broadcast on BBC Radio 4 on Tuesday 17 March at 4pm