If it was meant to highlight Boris Johnson’s tendency to flee the consequences of his actions, then the trick almost worked.

The Luxembourg prime minister’s decision to face the press with an empty podium where his British counterpart should have been, rather than let him move their press conference away from noisy protesters, has delivered the diplomatic burn that Xavier Bettel presumably intended. But in truth, it won’t change anyone’s mind.

Remainers may think Johnson looked pathetic, hiding from a few measly protesters; but plenty of leavers will just conclude that the sooner Britain leaves and stops being pushed around like this by EU nobodies, the better. It’s not how it looks that matters here, but what was actually said – and for once by the protesters, not the politicians.

For this wasn’t the usual crowd, and theirs was a message we don’t hear enough. Most were British people whose lives could be turned upside down on day one of a no-deal Brexit. Overnight, they fear losing the rights they had as EU citizens to everything from unrestricted travel across borders (an everyday reality in a country as tiny as Luxembourg) to residency in their own homes, to healthcare.

The picture is confused, since different EU countries are taking different decisions over how they will treat British citizens in the event of no deal. But documents released under Operation Yellowhammer suggest some British citizens living in the EU might face “substantial” bills for urgent medical care if they need it, while longer-term patients could see treatment disrupted. British consulates are braced for a flood of panicking people who don’t understand what advance arrangements they need to make, can’t navigate local bureaucracy or don’t speak the language well enough.

Not enough people are listening to what these protesters were actually trying to say

Such people are almost unique in getting precious little sympathy from either side of the debate. They are crudely caricatured either as rabidly rightwing pensioners sunning themselves on the Costa Brava, who probably backed Brexit anyway and are only now grasping the consequences, or as Dordogne-dwelling chatterati hopelessly out of touch with how their compatriots back home feel. But the estimated 1.3 million Britons who live in the EU aren’t cartoon characters. They’re human beings, who moved abroad for love or work or the chance of a better life, and never expected to be hung out to dry for it. No wonder they are taking it personally.

Those whose lives are now up in the air range from students at European universities and people who moved with their companies for work, to dual-nationality couples who wanted to let their kids experience both sides of their heritage, or people who worked hard all their lives and dreamed of retiring somewhere the cost of living was cheaper. They just grasped the opportunities we were all given in an era of open borders, and which until recently nobody expected to lose, only to be caught in the political crossfire.

For there is nothing inevitable about them becoming casualties of Brexit, even now; in any sensibly negotiated deal to leave, their rights could be protected, and so could those of EU citizens here. Their chronic anxiety is a direct consequence of Johnson’s decision to put no deal – and with it no visible safety net – back on the table.

Some may simply choose to come home to Britain, but for others it won’t be that easy. Those who married EU citizens worry about whether their partners would be allowed in, or whether families would be separated at the border. And all of them are trying to make decisions in the dark, still unsure whether in six weeks’ time they will be facing utter chaos, a smooth transition period with their rights intact, or just another extension and more uncertainty. In the circumstances, the scandal isn’t that a handful of protesters spoilt Johnson’s press conference, or that Bettel’s handling of it was ungracious. It’s that not enough people are listening to what these protesters were actually trying to say.

• Gaby Hinsliff is a Guardian columnist