If Britain votes to leave the EU, it will not be because of the threat of eurozone ganging up on the UK, disputes over the role of national parliaments in bloc decision-making, regulatory threats to the City of London or concerns over competitiveness. It will be because EU membership has become synonymous in many voters’ minds with uncontrolled immigration.

David Cameron, UK prime minister, has made curbing migrants’ access to benefits a central plank of his drive to renegotiate Britain’s membership of the EU before a referendum promised by the end of 2017. And, indeed, net migration to the UK has picked up strongly in the past couple of years as the country’s economic recovery has gained momentum. But, contrary to colourful British press warnings of “invasion” and “swamping”, it has not been exceptional in an EU context over the past 15 years. Net migration to Spain and to Italy was higher than to Britain over this period. The share of Britain’s population comprising non-Britons is not out of line with comparable EU countries.

Britain also has a strong record of integrating migrants, suggesting UK employers, both public and private, are more open to giving jobs to people with foreign qualifications than their counterparts in many other EU countries. This helps explain why Britain attracts more skilled immigrants than fellow member states. Proportionally, EU migrants are also more likely to be employed than native Britons and are big net contributors to the UK’s public finances.

So why has EU immigration become so toxic? One reason is probably because UK workers’ real wages fell sharply be­tween 2008 and 2014. There is little evidence that EU incomers as opposed to a deep recession caused this, but in the popular mind there is a causal link.

Another reason is housing. Building in the UK has lagged behind demand for 35 years. Despite a gradual rise in completions in the past two years, Britain is still building a third fewer houses than it was in 2007, making the shortage more acute every year. Many blame immigrants, yet the real culprit is an egregious failure of public planning policy.

Proportionally, EU migrants are also more likely to be employed than native Britons and are big net contributors to the UK’s public finances

Migrants are also blamed for putting pressure on the National Health Service and schools. But the problem is again public policy: the supply of public services is too slow to respond to increased demand. This is partly because of a government squeeze on spending. It also reflects an institutional problem: the tax benefits of immigration flow to central government, which is slow to compensate the NHS or local education authorities for the costs of providing the additional public services required.

A final factor behind rising hostility to immigration is the diminishing social status of the white working class. This group is now the worst educated in the country, as well as the most likely to be in low-paid work and competing for scarce social housing. Britain has an admirable record of integrating mig­rants but is weak at addressing the problems of their poor white neighbours.

Both the governing Conservatives and the opposition Labour party have found it expedient to scapegoat migrants rather than address the chronic policy failures driving hostility. By focusing on “welfare tourism” and on the purported need to limit incomers’ access to social housing, the government has deflected criticism of its austerity measures and pressures on public services. The price of such short-sightedness has been high: it has put in jeopardy the country’s EU membership, which confers far more benefits than costs on the UK.

The tragedy is that this situation could have been prevented, had the two leading parties shown some leadership by refusing to link immigration with social and economic problems, and by facing down populist sentiment in the media rather than pandering to it. But that would have required them to be serious about addressing the country’s supply-side problems, and neither has shown much stomach for that fight.

Simon Tilford is deputy director at the Centre for European Reform.