Brexit means Brexit. But does “controlled immigration” mean “free movement of labour”? David Davis’s remarks this week suggesting that migrants will be needed in the UK workforce for years to come shows that the Tories are not entirely united on what a Conservative immigration policy should like. The prime minister may have moved her party firmly into Ukip territory but economic reality is beginning to bite back.

And how is Labour doing? Well the other day, I was paid the biggest compliment of my political life. It was from a stallholder at Newcastle’s historic Grainger Market, which sells everything from turnips to Doc Martens, who said he’d heard me on the radio and I was “proper Labour, not old Labour or new Labour but proper Labour”. I’d been talking about immigration policy that week, and I hope that is what inspired his comments. Because it’s certainly true that we need a “proper Labour” immigration policy.

Many colleagues, whether on the frontbench or not, have struggled with immigration and Brexit. Certain colleagues have moved to swiftly reject freedom of movement in principle and practice.

Others have tried to demonstrate a tough new approach to immigration without actually undermining the principle that people should have the same freedoms as goods.

And some have sought to address what they consider the underlying causes of the fear and anger expressed by so many here which targets, often viciously, migrants.

But what is a proper Labour response? Both principle and pragmatism have their place. I have argued previously that Labour should dare to defend freedom of movement, both as a matter of principle and credibility. Are we really going to knock on the doors of council estates and say: “Oh yes, it was a big mistake to let people in. Actually we didn’t realise these people were immigrants. We do now, and of course there are too many of them, so we’re not going to let any more in, promise. Please vote for us.”

Saying there are too many immigrants is basically saying “Go home” to the nurse caring for your gran; to the student studying for their masters at your local university; to the pupil popular with his classmates in your local school.

Immigrants are people. Saying there are too many of them is why Donald Trump senses an ally in the UK as we shut the door to the EU. It’s also why hate crime has risen nearly 60% in the wake of the Brexit vote and is expected to rise again when article 50 is triggered.

Such negativity is all the more surprising when all the studies show that immigration brings overall positive benefits for the economy and society. Immigrants raise skill levels and generate employment opportunities for others. On average, EU migrants pay in 30% more via tax than they take through benefits and public services.

However, it is true that any increase in population puts additional pressure on local services such as housing and schooling. As well as the local economy, social infrastructure can also be affected. Migrants are less likely to speak English, which can be reductive for local communities. In London, where no one speaks to each other anyway, this may not matter. In Newcastle, where the bus stop is our equivalent of the water cooler, it does. We need strong communities to foster shared values, and communication is an important part of that.

The last Labour government brought in a migration impact fund to try to ensure that the benefits of immigration were shared more fairly, but this was torn up by David Cameron. That is feeding the anger today, destroying trust in politicians and stoking hostility to migrants.

So what is a principled, pragmatic response? It may be pragmatic to accept short-term changes in the implementation of free movement rules, for example by adopting existing EU powers. My Labour colleague Alison McGovern has proposed strengthening workers’ rights to prevent the exploitation of migrant labour, while Lisa Nandy has highlighted the need to improve education so that British workers can compete with higher skilled migrant workers.

But we must not concede the principle. If we say that we want your goods but not you, we downgrade people and make them of less worth than the things we produce. The Labour party is about the value of and the people whose labour it is.

It is particularly important to remember that as the fourth industrial revolution gathers force. Wages in Romania have steadily risen since the country joined the EU, on average by 4.1%, reducing the incentive to move to richer countries. Wages of robots on the other hand remain a stable zero. And according to the Massachusetts Institute of Technology one new robot per 1,000 workers reduces wages by up to 1.6%. Robots have absolute freedom of movement, and when they come to your workplace, who you gonna blame?

Principles endure; pragmatic solutions change. If Labour shouts our pragmatism from the rooftops – as some are doing – we appear unprincipled and untrustworthy. If Labour whispers our principles in the spaces between the noise – as others are – we appear lethargic and untrustworthy. We need an approach which shouts both our principles and our pragmatism. Now that is what I call “proper Labour”.