It is a favoured trope of Nigel Farage and his fellow-travellers that the “Westminster elite” doesn’t want to talk about immigration. And it was true once. Back in the day when he was still calling himself a Tory moderniser, David Cameron used to tell friends that he was cautious about “going there” because “you don’t know where it is going to run away to”. He made a reckless promise on numbers during the last election, the breaking of which has further fuelled public distrust, but immigration did not feature that prominently in the Tory campaign of 2010. As for Ed Miliband, when he came to the leadership of the Labour party, immigration was not the first issue on his mind. I doubt it was even in the top 10.

Neither man anticipated what has happened over the course of this parliament. Immigration has steadily barged up the hierarchy when pollsters ask voters what concerns them most to the point where it now rivals the economy as the number one subject. The associated rise of Ukip is causing fissures and febrility within both the big parties. A subplot of the recent turbulence around Ed Miliband has been a long-running argument at shadow cabinet level about how far Labour should go in “toughening” its posture on immigration. One of the greatest sources of anxiety among Labour backbenchers is the fear that immigration is mainly responsible for their leakage of votes to the Kippers. There was an unusually full-frontal attack on Ukip from Mr Miliband in his recent “fightback” speech that will be followed this week by more detail from Labour about its policies. Says one senior Labour figure: “Talking about immigration has become a touchstone of whether you are listening to voters.”

As for Mr Cameron, the man who used to express himself wary of “going there” has been sucked into a populist bidding war with the Farageistes. It didn’t save the Tories from a thumping at the Clacton byelection and it won’t spare them from another bloodied nose at Rochester and Strood. The bookies aren’t waiting to hear from the returning officer. They have already started paying out to punters who bet on a Ukip victory in Kent. We are told that the prime minister plans to respond with a supposedly definitive speech about how he would seek to renegotiate the rules on freedom of movement within the European Union. That is a further slippage from his original position that he would not start detailing his negotiating position in advance. Once again, he risks raising expectations that he cannot meet and making promises that he cannot deliver.

All the traditional parties are struggling to find ways of talking to the voters about immigration that manage to stay true to their principles and faithful to Britain’s national interests while also being practical, deliverable, trusted and capable of mobilising broad consent. Dismissing public anxiety as just irrational, which has been the vice of some liberals, doesn’t work. Stoking public anxiety, by ramping up “tough” rhetoric and making promises that can’t be kept, is even worse.

Some timely guidance on a better approach is at hand from the innovative thinktank, British Future, led by Sunder Katwala. This week, it will publish the results of a three-year investigation into public attitudes towards immigration. It contains a lot of fascinating new polling that drills deeper than the questions usually used for weekly surveys for newspapers. How to Talk About Immigration also draws on a lot of structured conversations with voters across the country from Glasgow to Southampton, Leeds to Cheltenham, Bolton to Cardiff. The good news is that public attitudes are not as simplistically hostile as is often assumed. Xenophobes who think most voters share their toxic views and liberals who fear that to be the case are both wrong. The majority of voters have a much more complex set of attitudes that are not being reflected by the party political slugfest. “When they talk about immigration the public is moderate, not mad,” finds this investigation. “Most people aren’t desperate to pull up the drawbridge and stop all immigration, nor are they crying out for more of it. Instead they’re somewhere in the middle: worried about the impacts on jobs, public services and on the ‘Britishness’ of our culture; but aware of the benefits to our economy.”

There is what the study calls “the rejectionist minority” who “would close the borders, or even send all migrants back”. But they are not representative of most of us. The “slam the door” segment of the electorate makes up around a quarter of the population, a finding that chimes with the private belief of some Ukip insiders that 25% is the absolute maximum ceiling on their potential national support at a general election. Nigel Farage likes to claim that he speaks for “the silent majority”. He is actually the megaphone of a noisy minority.

Compared with the typical voter, members of this rejectionist minority are more likely to have left school at 16 and they are less affuent. They are older as well. Several surveys have now established what Bobby Duffy, of the pollsters Ipsos MORI, calls this “counterintuitive” trend. The young, who might be expected to be more resentful of competition for jobs, have become more positive about the economic impact of migrants. It is older cohorts who are near or in retirement who have remained negative. For all its current successes, this heavy reliance on the older voter suggests that Ukip’s long-term prospects are not promising unless the party finds a way of broadening its base. The phobic vote is a dying vote. That’s the bad news for Ukip.

The bad news for liberals on immigration is that they are also in a minority. There is a slice of the population that is happy with current levels of immigration, the most internationalist of whom think that a world without borders would be an attractive ideal. As you might expect, members of this group tend to be younger and more ethnically mixed than the average voter. They are more likely to have a higher education and to live in London or another big city. The liberals represent around a quarter of the public.

The key to winning the political debate on immigration lies with the half of the population whom the study characterises as “the anxious middle”. They are unimpressed by how governments have managed migration. They say they like the idea of targets for numbers, but they don’t trust anyone’s promises. The public doesn’t have much trust in any politician on any issue at the moment. But there is a highly enjoyable paradox that an immigrant who has been here for 15 years and has become a British citizen is more trusted by more people “when they talk about immigration” than any of the party leaders, Nigel Farage included.

That highlights an important point about “the anxious middle”: they do not have anti-migrant views per se. Most express themselves proud of Britain’s tradition of offering a sanctuary to refugees from conflict and persecution. A lot of this group can appreciate the value to Britain of attracting talented people to these shores. At the same time, they express a great deal of anxiety about pressure on services. The sobering finding for liberals is that their arguments don’t have much traction on these concerns. The way some liberals talk can even aggravate voters’ discontent. Liberals often contend that they can win this debate if only the public are told the facts: that there are fewer migrants here than people think and those who are here are good for the economy and that they pay in more to the welfare state than they take out. The study finds that messages designed for “elite policy advocacy” don’t work outside the boardroom, government department or seminar room. People aren’t persuaded that immigration is a boon for the economy as a whole if they don’t feel it in their personal pocket. One workshop with voters in Coventry presented them with a lot of official figures on the economic benefits of immigration. The participants ended the session with greater anxieties than they had begun with.

The public does have a very poor grasp of the scale and nature of immigration. In Ipsos MORI’s most recent survey on this, the average guess at the proportion of the UK population that is foreign-born was 31%. The official estimate is 13%. Yet simply telling voters they are misinformed risks hardening attitudes about immigration among the “anxious middle”. People don’t like to be called stupid. And they really don’t like it when liberals imply that anyone who disagrees with them on immigration is either a racist or the dupes of racists.

The majority of public opinion has an attitude towards immigration that is utilitarian. Nearly two-thirds signed up for the proposition: “Immigration brings both pressures and economic benefits, so we should control it and choose the immigration that is in Britain’s best economic interests.” Liberals shouldn’t be responding to the current climate with despair. They can have confidence that Farageism is not representative of most of the public. They need to combine that with the humility to recognise that most of the public aren’t liberal either. Patronising and pandering doesn’t work. It is plain-speaking pragmatism that most voters are looking for. And someone to trust.