A powerful cross-party alliance including former Tory foreign secretary Malcolm Rifkind and deputy prime minister Nick Clegg is calling for an urgent fightback against spiralling anti-European sentiment as a new four-nation poll suggests the UK could be heading out of the EU.

The landmark survey of more than 5,000 voters in the UK, Germany, France and Poland finds British people far more hostile to the EU and its policies than those in the other EU states, and strikingly low support for British membership among people on the continent.

At the same time, the total numbers of people in Germany and France who support giving Britain a special deal on membership to satisfy British opinion are heavily outnumbered by those who oppose doing so, which suggests that David Cameron may struggle to achieve his hoped-for tailor-made arrangement for the UK.

Testing cultural opinions, the poll finds very few British people choose to describe themselves as European. In other EU nations, enthusiasm for the concept of Europeanism is far higher.

Opinium found that just 26% of British voters regard the EU as, overall, a "good thing" compared with 42% who say it is a "bad thing". In Poland 62% say it is a good thing and 13% bad; in Germany 55% good and 17% bad, and in France 36% good and 34% bad.

When asked about the UK's contribution to the EU, there is little enthusiasm among our partners, and little to suggest they will go out of their way to keep us in. Just 9% of Germans and 15% of French people think the UK is a positive influence on the EU, with more Poles, 33%, taking that view.

Only 16% of Germans and 26% of French people back the idea of a special deal being struck for the UK. Cameron has said he intends to renegotiate the UK terms of entry and hold an in/out referendum if he wins a majority at the next election, offering the new arrangement to the British people in a referendum.

The idea of Britain leaving the EU does not appear to worry our European partners unduly. Just 24% of French voters said a UK exit would have a negative effect, compared with 36% of Germans and 51% of Poles.

Rifkind said: "There needs to be a serious debate about the real benefits of – as well as the real problems about – British membership of the EU. Without it we could do serious damage to Britain's interests."

Clegg said next year's European elections represented a key test and attacked those intent on taking Britain out of the EU. He said: "Everybody knows the EU needs reform. But simply carping from the sidelines and flirting with exit undermines British leadership in the EU, fails to deliver reform and leaves Britain increasingly isolated. The debate about Europe is no longer about who is for or against reform – everybody agrees on that – it is between those who believe we can lead in the EU and those who want to head for the exit.

"That's why next year's elections will be so important: the Liberal Democrats will be the leading party of 'in'. It's time we challenged Ukip and large swaths of the Conservative party who want to betray Britain's vital national interest by pulling us out of the world's largest borderless single market, on which millions of jobs depend."

Labour MP and former Europe minister Peter Hain urged pro-Europeans to stand up and fight: "This is a wake-up call for British pro-Europeans that Britain – especially if the Tories win the next election – is heading for an exit from the EU which would be an utter disaster for British jobs, prosperity and influence in the world. But it is equally a wake-up call for the Brussels Bubble, which is totally out of touch with Europe's citizens."

The poll shows concern about immigration to be almost as high in France as in the UK. In Britain, 64% of voters think the EU's immigration policies have a negative effect; 59% say the same in France.

It also reveals that more UK voters feel an affinity with the US than with their European neighbours, whereas our EU partners tend to choose other EU nations. When asked who they would generally support on occasions when there was a disagreement between the US and EU countries, 37% of UK respondent said they would tend to support America; just 10% would generally side with Europe.

British people are not negative about everything the EU does: 54% think free movement rules are good for tourism against 6% who think the reverse. There is also strong endorsement for free-trade benefits. Nearly half of those polled say the absence of customs controls and tariffs on goods and services is an advantage. Only 10% see free trade as a disadvantage.

Ukip leader Nigel Farage said: "This is a fascinating and comprehensive study into the relative relationships between countries within and about the EU. We, on these islands feel, due to our history as a globally trading nation, much more at home with our cousins in the Anglosphere than we do with our friends on the continent."