Cambridge, 2014. Flickr/Trojan-Llama. Some rights reserved.‘Sheffield’, he echoed with a snort, when I said where I was from.

Someone tried to rescue him, ‘People can’t help where they come from.’

‘They can move,’ he answered.

This exchange was one of my first as an undergraduate at Cambridge University. My interlocutor had spent most of Fresher’s Week mentioning that he’d travelled to all but one continent. (Antarctica was planned for the summer.) Paris was a regular hop, but it transpired that Cambridge was the furthest he’d been outside London. He was not white, but had a professional father, and attended a top London grammar school—the kind that had Boris Johnson bestow prizes on practically every leaver.

“They can move” is not just a naïve nineteen-year old’s retort. It has been the policy position of the, now-marginal, political centre. Recurring op-eds reason that public investment in post-industrial areas is, as some put it to me in person, pissing up the wall. Instead, “they” should move to growth areas, like Cambridge. The position was apparently apolitical; it would signal the speaker ‘got’ economics.

But understanding why people in post-industrial areas haven’t moved, out of their local area, or up social classes, is necessary to explain this disproportionately white, working class group’s pro-Brexit politics. The politics set, against the consensus of economists, to end EU freedom of movement.

We have to start by recognising that most people don’t want to move. People are attached to the place they grew up in— even when they know it’s ugly, grey, and jobless. They are attached to their family—no matter how apparently ‘troubled’ it is. They identify with their region and their class—all the more when others mock it.

But let us also consider why some people are not free to move. People who move place or class (and the two often go together) tend to have subtle advantages. To move takes resources—and not just a travel fare or formal education. Moving takes a mind for complexity, an imagination for ambition, a stomach for homesickness—and a readiness to risk rejection.

This is how, for all the discrimination they experience, immigrants and their children in England can have resources those left-behind lack. At school, ethnic minority working-class boys fare better than their white counterparts, while more diverse inner London schools do better than majority white ones. Experiences vary, especially by class, but perhaps minorities have pushier parents who didn’t leave home for nothing. Perhaps, with family abroad, they have a mental map that orientates them in a global world; tastes that readily cross cultures. So total is England’s class system, categorising people on every preference and pronunciation, you can be lucky if, by ethnic difference, you don’t fit neatly into it.

So total is England’s class system, you can be lucky if ... you don’t fit neatly into it.

Migrant’s advantages are at the margins. But the competition isn’t only for jobs or wages. It’s also for deservingness and dignity. Jealousy never excuses xenophobia. But this potent mix of emotions —fear, inadequacy, resentment, distrust, and disgust— can help us comprehend it.

It also helps explain why the liberal response, to point out migrants’ net benefit to economic growth, fails. Why would it help to hear that many of our doctors, scientists and lawyers—professions still largely closed to those from lower classes—are not white or British? Or that the Polish builders are harder working or the Spanish sandwich-shop servers have a better attitude than native workers? It only shows them up.

Xenophobia does not flare up when migrants are failing to meet national cultural norms — it flares up when the native-born population is. Slow-burning anti-Semitism became a wild fire when Jews took up new opportunities that Germans were poorly prepared for and resisted.

Those who have the “mobility mindset” – the literal ‘get-up-and-go’ – may be accepted, even celebrated. But those who don’t, the story continues, lack character. Character, as if it were an essential property of an individual untouched by socio-economic experience. Character, as if the recent shift in its meaning—from acting morally to acting in whatever way economically successful people do—were seamless.

The reaction to this narrative, as moral as it is economic, has been to reverse it. To claim the superiority of the worst stereotypes of white working class cultures. To rebuff movement. The Conservatives, borrowing from UKIP, have successfully honed this inverse snobbery. Labour, long ambivalent about migration and social mobility, has also reverted to it.

We can hear a claim for class, as well as national-ethnic, identity in support for Brexit. But it emerges not from pride, but from shame. The jealous group, like the lover, is only more likely to see their fears, of humiliation and abandonment, come true. For it is not that the advantages of the mobile are never liberating or life-enhancing. It is that they are what inequality denies.

Rejecting mobility can never challenge the economic injustice – or the psychic injury – of class and regional inequality, because it essentialises it. It shares the conservative idea that status and skills are intrinsic, inherited properties; that people should stay in their place. The idea that some things simply ‘aren’t for the likes of us’ – things that include higher education, ownership, and environmental and human rights protection – has always been the convenient way to conserve power.

Movement has been an obligation for people seeking decent jobs and dignity, and one they cannot fulfill. Movement brings loss. But so too, in an unequal status-quo, does stasis. We deny people dignity when we deny them the ability to move – place, class, or politics: when we pat the working class on the imagined flat-cap, with false promises of bringing back the fifties.

The white working class doesn’t have true freedom of movement. But they are not the people who will benefit from its end. England’s elite power remains rooted in inherited title, property, and press ownership. It dismisses demands for public investment – and tax on unearned wealth that might pay for it – as the politics of envy. But it is envy that its politics has fed off: a conservative politics that blocks movement, not least towards a better future for the white working class.

This piece was first published on February 1 in the LSE Brexit blog.

