The idea that Peterborough feels like the future may seem unlikely. But it’s true: this bustling city with a population of 190,000 really does feel as if it’s on the social and economic cutting edge.

Brand new housing developments extend into the distance, as does rush-hour traffic. On its southern edgelands is Kingston Park, a surreal clump of vast warehouses and distribution sheds that includes an unbelievably large Amazon “fulfilment centre”, and one empty newly built structure, seemingly waiting for another multinational to fill it. Throughout the day, walkways and roads nearby are smattered with people in hi-vis jackets, living the zero-hours life, and trying to make the best of it.

Talk to them – Hungarians, Lithuanians and Bulgarians – and you’re reminded of two things about this part of England: the presence of large communities from central and eastern Europe; and the fact that their long-hours, low-wage work sits behind parts of the economy we all tap into every day.

In Peterborough it is warehousing and distribution; in nearby Wisbech, the food industry. Meanwhile, modern lives flip between trips to Aldi or Lidl, and online shopping sprees. Make no mistake: it is these people’s toil that the whole circus depends on.

As the article 50 debate grinds on, Downing Street has reportedly been warned that “at least half a dozen” Conservative MPs may join those from other parties to call for a legal guarantee that the 3.3 million EU nationals living in the UK will be able to stay. David Davis, the Brexit secretary, says nobody would be “throwing people out of Britain”, though his department’s white paper published on Thursday offers nothing more concrete than the statement, “We want to secure the status of EU citizens who are already living in the UK … as early as we can.”

Until negotiations are under way, certainty is obviously a non-starter. Besides, whatever the eventual legal outcome, the conversation about Brexit – which was backed by 61% of Peterborough’s voters – may already be sending out the cultural signal that migrants should either go home or not bother coming. Whatever, talk to such people here about their future and the conversation instantly pushes abstract matters of negotiating stances to one side: questions about the EU are raw and visceral.

In a country as class-bound as ours, the idea of some kind of British dream might seem incredible. But for many of the people who have come to this part of England from the former communist bloc, the idea of the UK as a land of opportunity is real. And for some Polish people in particular, it seems to have come true: you start working insane hours in a minimum-wage job, then move up to a middle-management position, and eventually buy a house. Starting a small business may be an option. Meanwhile, whatever your career status, you will have kids who are effectively British – or, as everyone here puts it, English.

How odd that such a quintessentially Thatcherite ideal is being put under threat by her political heirs. “There are fears that they might chase us out of here, fears of deportations,” says one woman working in a Polish-owned food shop. “But life goes on.”

I’m told there have been rumours that, with the formal triggering of article 50 in March, EU citizens will suddenly not be allowed into the UK, even if they are coming back after a trip home. “It is stressful,” says Petya, who came to the UK from Bulgaria. “People are not sure if they can leave … or even if they can afford to stay.” The falling value of the pound also plays on people’s minds.

In Wisbech, Lionel Sheffield is the boss of a firm, Rapid Recruitment, that supplies largely eastern European workers to local businesses. He reckons that applications to be on his books from EU countries fell 50% immediately after the referendum, and are now 25% down year-on-year. He fears a “precipice”.

Try coming up with a model of consumerism that avoids huge seasonal fluctuations

David Orr, who owns local packaging firm Fencor, uses temporary workers whose numbers fluctuate with his business cycles. He’s written to ministers asking how this will work post-Brexit, and says the government is “not even bothering to listen”.

Orr says he has been pointed towards so-called tier 2 work visas, which involve formally becoming a “sponsor” approved by the UK Border Agency, a licence costing £536, and the obligation to put adverts in jobcentres for at least 28 days. He recently wrote to the home secretary warning that “the outlook at present looks frighteningly like the autumn of 2009 – the start of the last recession – [but] the big difference is that this time most of the economic damage will be self-inflicted”.

Left-leaning people may read about such cases and respond with moralistic sighs. Why, they may wonder, are businesses reliant on all that agency labour? But try coming up with a model of consumerism that avoids huge seasonal fluctuations. As Brexit may yet prove, coming down hard on this part of the economy would lead to lots of businesses going under.

It may turn out that the EU’s key contribution to Britain’s economy and society over the past 15 years was not the high-flown stuff about European cooperation and internationalism, but the way that it provided a huge pool of workers who would do jobs most British people would balk at, and thereby sustained a fragile mess of stagnating wages, skyrocketing credit, cheap food and consumerism-as-culture.

Millions of leave voters have experienced the magical benefits of all that just as much as those who voted remain. And if the whole model starts to unravel, their howls of dismay will be just as loud. It would be a very British outcome: in the land of having your cake and eating it, proof that if you play fast and loose with the people who do the baking, the fun soon stops.