The leaked proposals on post-Brexit immigration give us a useful insight into the government’s thought processes. The desire seems largely to be about addressing “public concern” over pressure on public services, depression of wages and displacement of UK workers: that is, issues of perception, whether or not they are real.

That policy is driven by an obsession with meeting the net immigration target of “tens of thousands”: despite it being continually exceeded since the Conservative pledge was made seven years ago; despite its dependence on levels of emigration over which the government has absolutely no control; and despite the inclusion of overseas students in the net immigration numbers whose contribution we now know to be massively overstated.

The document makes much of the distinction between skilled and unskilled labour and between “hard” and “soft” skills. Bankers, engineers and footballers are welcome, but not hotel receptionists, care workers, chefs or cleaners. By implication, unskilled jobs should be reserved for British workers. But it isn’t clear where, with record levels of employment, the reserve army of unemployed, unskilled Britons is currently billeted.

At the heart of the politics of immigration is the belief, repeated by Theresa May as a fact, that immigrants, especially unskilled immigrants, depress wages. At first sight the argument seems plausible – and undeniably there is low-wage competition in some places. But there is no evidence that this is a general problem. When the coalition embarked on its review of EU competences in 2013, I commissioned a range of reviews and studies to establish the facts. They showed that the impact on wages was very small (and only in recession conditions). By and large, immigrants were doing jobs that British people didn’t want to do (or highly skilled jobs that helped to generate work for others). This research was inconvenient to the Home Office, which vetoed the publication of its results. I have now written to the prime minister to ask her to publish them as part of the current public debate.

In support of the conclusions, the All-Party Parliamentary Group on Migration’s latest report – Brexit: Beyond the Highly Skilled – the Needs of Other Economic Stakeholders – highlighted several startling cases. For example, one evidence-giver recalled that they recently hired 53 individuals, of whom 11 were UK nationals and 42 EEA nationals. Of the 11 UK nationals, eight left on their first day, saying the work was not for them, as it was too repetitive and the environment was too cold. All 42 EEA migrants were still there at the time of the report’s publication. Stories such as this are repeated up and down the country.

Moreover, there is no single definition of a low-skilled worker. The Office for National Statistics uses a skills classification formula that is based on the time necessary for someone to learn how to perform the task required of them, whether through formal qualifications or on-the-job experience. This approach disregards the “soft” skills that for some jobs are crucial, as in health and care. An immigration policy based on ministers’ distaste for people with low skills is going to end badly. Indeed, one of the ironies of the immigration debate is that the Conservatives, who normally take refuge in the wisdom of markets, have an almost Soviet enthusiasm for centralised government direction and the wisdom of bureaucracy when it comes to migration.

All this is not to deny that immigration brings costs as well as economic benefits. The most obvious, if least discussed, is the impact of rising numbers of people on the housing market, where supply is constrained. House prices and rents are bound to rise. Yet the critics of immigration rarely raise this issue, perhaps because the beneficiaries are generally older owner-occupiers (who tend collectively towards Brexit) and the losers are younger people in cities such as London (who tend collectively towards remain). Here, as elsewhere, it is perception that is driving immigration policy; not reality.

Nor is it satisfactory simply to ignore the perceived or real problems around immigration; the political class has lost respect and authority by doing just that. There clearly is need to take investment in skills training and continuing education for British workers much more seriously. The frustrations over public and private sector pay are real, even if immigrants are not the main reason for wage stagnation. The anecdotes about immigrants in doctors’ surgeries reflect a wider problem of stress in primary health care. And immigration has compounded an underlying failure in housing supply.

The political temptation to play to the gallery rather than address these underlying problems is, sadly, all too real.

• Vince Cable is leader of the Liberal Democrats and former secretary of state for business