Level of far-right support could outstrip that in France or Holland, says poll for Searchlight

Huge numbers of Britons would support an anti-immigration English nationalist party if it was not associated with violence and fascist imagery, according to the largest survey into identity and extremism conducted in the UK.

A Populus poll found that 48% of the population would consider supporting a new anti-immigration party committed to challenging Islamist extremism, and would support policies to make it statutory for all public buildings to fly the flag of St George or the union flag.

Anti-racism campaigners said the findings suggested Britain's mainstream parties were losing touch with public opinion on issues of identity and race.

The poll suggests that the level of backing for a far-right party could equal or even outstrip that in countries such as France, the Netherlands and Austria. France's National Front party hopes to secure 20% in the first round of the presidential vote next year. The Dutch anti-Islam party led by Geert Wilders attracted 15.5% of the vote in last year's parliamentary elections.

Anti-fascist groups said the poll's findings challenged the belief that Britons were more tolerant than other Europeans. "This is not because British people are more moderate, but simply because their views have not found a political articulation," said a report by the Searchlight Educational Trust, the anti-fascist charity that commissioned the poll.

According to the survey, 39% of Asian Britons, 34% of white Britons and 21% of black Britons wanted all immigration into the UK to be stopped permanently, or at least until the economy improved. And 43% of Asian Britons, 63% of white Britons and 17% of black Britons agreed with the statement that "immigration into Britain has been a bad thing for the country". Just over half of respondents – 52% – agreed with the proposition that "Muslims create problems in the UK".

Jon Cruddas, the Labour MP who fought a successful campaign against the British National party in his Dagenham and Rainham constituency in east London, said that the findings pointed to a "very real threat of a new potent political constituency built around an assertive English nationalism". The report identified a resurgence of English identity, with 39% preferring to call themselves English rather than British. Just 5% labelled themselves European.

Earlier this month David Cameron delivered a controversial speech on the failings of "state multiculturalism". The speech was seized on by the anti-Islamic English Defence League, which said that the prime minister was "coming round" to its way of thinking. BNP leader Nick Griffin also welcomed the speech as a sign that his party's ideas were entering "the political mainstream".

The poll also identified a majority keen to be allowed to openly criticise religion, with 60% believing they "should be allowed to say whatever they believe about religion". By contrast, fewer than half – 42% – said "people should be allowed to say whatever they believe about race".