“Let’s pretend we were inventing the world from scratch today and you decided you were going to have borders,” said Benjamin, a migrants’ rights activist, to me a few years ago in a cafe in London. “On what basis would you make the decision about who gets to move and who doesn’t get to move through borders? Surely you wouldn’t make them in the way the world is constituted today?”

We live in a world where your gender, race and class shape whether you can move and how you’re treated if you do. While many of us strive to change that, too many of our politicians are entrenching that system and making things worse. This is precisely what the Conservatives’ immigration proposals would do.

This is a hardening of an immigration system that’s already built on elitism

While a lot of the plans themselves aren’t new and are lacking in details, how they’re packaged tells us a lot. One of the government’s headline policies is that people coming into the country will “have to speak English”. This dog whistle is depressingly familiar: a decade after New Labour announced a “points system” that included English tests, in 2015 the Labour leader Ed Miliband used it when he said speaking English was “something we should expect from everyone who comes here”. For decades politicians have turned to the English language as a symbol to show they are willing to be tough on immigrants and say “hard things” when they need to.

It ceases to matter that most migrants already speak English, that language diversity is positive for all kinds of reasons, or that someone’s ability to move shouldn’t be contingent on their knowledge of a particular language (there’s far less focus on this when it comes to English speakers not knowing other languages in countries all around the world). Some of the same people who might encourage their children to live and work abroad for a year to improve their French or Mandarin boast about policies that’ll decide who can and can’t come into the country on the basis of their English skills. Learning other languages is something we should all have the time and opportunity to do – one survey suggested 62% of British people only speak English.

But we all know these proposals are nothing to do with that at all because they don’t even pretend to be. Instead, they’re based on deeply xeno-racist grounds – that is a racism “directed at the displaced, the dispossessed and the uprooted”. Politicians claim English language skills are necessary for “integration”, a highly racialised term that swiftly becomes forgotten when the focus of these plans is on temporary visas. That is, the documents that put a time limit on how long people can be here, thus getting in the way of relationships they’ve built, friends they’ve made and people they’ve fallen in love with. How are immigrants supposed to “integrate” when they never know for sure how long they will be allowed to stay? The logic for one immigration policy is often the opposite of another.

Quick guide Impact of proposed immigration points system Show Hide How it works Bulgarian welder, holds A-level equivalent, has job offer for £26,000 a year, does not speak English. Now: Able to work in the UK under free movement rules.

From January 2021: The worker scores points for a job offer, salary over £25,600, educational qualification, and working in a shortage occupation – meaning a score of 80 points, 10 more than the 70-point threshold. Ticks two of the three mandatory boxes for entry to the country – a job offer and job at appropriate skill level. But falling short on the third compulsory condition for entry of speaking English rules the welder out and they cannot come into the UK. Sri Lankan production manager, has job offer for a salary of £28,000 a year, holds A-level equivalent, holds a PhD in a Stem subject, speaks English. Now: Eligible courtesy of the requisite educational qualification of degree or over.

From January 2021: The worker earns less than the £34,000 “going rate” for their profession, meaning that they must pick up 70 points elsewhere to be eligible. They are not in a shortage occupation and so score zero on that point – but succeed nonetheless with 20 points for a job offer, 20 points for their A-level equivalent, 10 points for English, and 20 points for a PhD in a Stem subject – a total of exactly 70. Italian waiter has job offer in a hotel at £20,000, has languages degree and fluent in English.

Now: Able to work in the UK under free movement rules.

From January 2021: Is eligible to enter on the three mandatory conditions – job offer, speaks English and has met education threshold. Picks up 50 points. But scores zero for salary, zero for shortage occupation, does not have a PhD and cannot come into the country.

Yet, it’s all presented as perfectly sensible. In a fit of supposed egalitarianism, Boris Johnson announced at the start of the year that he wanted to make an immigration system that was “people over passports”. It’s a way of wrapping up anti-immigration policy as reasonable while laying the ground for regressive announcements. It’s fair because there’s not preferential treatment for EU citizens, the defence goes. While fortress Europe’s borders and the UK’s restrictive, violent and highly racialised non-EU policy are certainly not worth defending, these new plans will likely spell poorer rights for a lot of people.

The “people” whom Johnson wants to put ahead of passports isn’t everyone; it’s the high earners, the “high skilled” or those with PhDs. In other words, in a system where racism and classism are deeply embedded in our borders – immigration for the rich or those from white-majority countries is already easier – the government’s plans will likely deepen these inequalities. If you’re classed as “low skilled”, a visa will be difficult to come by; if you don’t earn £25,600, the same will be true. The aim, Priti Patel said, is not only to stop “low skilled” people from moving to the UK but to go after “economically inactive” Britons, by forcing business to retrain uneducated, unskilled locals. The “undercutting migrant” and the “British scrounger” once again side by side.

Will Boris Johnson’s immigration reforms work? That depends on the Home Office | Jonathan Portes Read more

This is a hardening of an immigration system that’s already built on elitism. If you really wanted to make this about people, there are a variety of straightforward things that could be done: reducing the astronomically high fees people must pay to apply to live here rather than increasing them, bringing back legal aid or ensuring there are safe routes of passage for people seeking asylum so they don’t have to climb under lorries to try to get here.

The scapegoating of migrants that has been a mainstay of the immigration debate for decades is still very much there. The ground laid for so much of this has been decades in the making. Whatever happens next, you can bet the government will still find a way to blame migrants for this country’s economic and social problems. Forcibly deporting people to countries they barely know or detaining them indefinitely – one of their big plans is to show they are “tough on immigration”.

Already too much of this discussion is from the perspective of business interests or based on the idea that people have “legitimate concerns” about immigration. But what’s often forgotten is that it is immigrants – human beings who happened to be born in a different country and have moved to the UK – who are at the centre of this policy.

• Maya Goodfellow is a writer and academic, and the author of Hostile Environment: How Immigrants Became Scapegoats