When Amber Rudd threatened to jail landlords renting homes to illegal immigrants at the Conservative party conference in 2016, she framed the policy as an act of political liberation. For five years, under the coalition, Theresa May had been “held back” by the Lib Dems.“Freed from the shackles”, it turned out, the new Prime Minister would be able to inflict organised cruelty anyone suspected of being illegal – which of course means people who looks or sound non-British, or are non-white.

Renting a flat, getting a minicab license, using the NHS – the routines of everyday life would become a filter against illegal migration. People whose status was doubtful would effectively be faced with a border patrol every day.

We’ve seen the results in the Windrush scandal. Rudd’s 2016 speech was designed to give permission to the xenophobia that had built up and burst through during the Brexit referendum campaign. As Rudd acknowledged, there was no way to target East and South Europeans as long as we were inside Europe, so the people targeted would be mainly from the Commonwealth and the Middle East.

Not only did the system set about destroying the lives of black Caribbean Britons who could not produce the right papers; the political system and most of the media ignored the issue.

I met Amber Rudd once, on Question Time, and she struck me as a genuinely liberal, civilised, humane and democratic politician. How could someone like that stand up at Tory conference demanding employers make “lists of foreign workers”? How could she be the one inflicting mental torture on Jamaican pensioners?

There is only one concept that explains it: colonialism. Yet the deep-rooted colonial attitudes through which white British people appoint themselves superior to those who were once their colonial subjects is the ultimate taboo issue in this country.

The Commonwealth heads of government meeting showed how the taboo is maintained. The Commonwealth is supposed to be a voluntary association of equals. Yet it was still “persuaded” to appoint the future King of England as its next boss. And when prime ministers from across the Caribbean asked to meet Theresa May about the Windrush crisis, they were initially dismissed as time wasters.

We need to understand what colonialism was, and what it did. Its roots were economic. Without the Portuguese, Dutch and then British mercantile empires, capitalism could never have taken off. The meagre financial systems and irregular surpluses inherited from late-stage feudalism could not, on their own, produce the vast wealth that accumulated by the 18th century in Lisbon, Amsterdam and London.

Since market-based systems require the formal equality of participants, the colonial empires had to be justified by categorising the human race. At the bottom were the indigenous peoples of places like Australia or New Caledonia, both formally classified by colonial governments as sub-human. Then there were black Africans, who were enslaved and around whom their white owners created a unique ideology of hatred. Then the brown peoples of Asia, seen as “cunning” or “lazy”, and incapable of achieving anything without a kick up the arse from a British soldier.

In order to justify the theft of their natural resources, the imposition of lop-sided trading rules, cartels, slavery, bonded labour and the rape of their children, colonial subjects had to be classified as subordinate human beings.

In colonial ideology, the problem was not cruelty of the coffee planter or the slave ship owner. The fallibility was all on the side of the subject peoples themselves – above all their alleged low intelligence, which the ethnographic science of the 19th century devoted itself to proving.

Aimé Césaire, the Matinique-born French philosopher and poet, once said that fascism was simply colonialism applied to Europe. What the Germans did to the Poles and the Jews, they had first practised on the Herero population of what is now Namibia. The light brown uniforms of Hitler’s original paramilitaries were military surplus from the Afrika Korps of imperial Germany, and hundreds of those who wore them on the streets of Berlin in the 1920s had worn them during the Namibian massacres of 1904-7, an event now recognised by the UN as genocide.

But what about the Brits? The most infamous atrocities of the British Empire include the 1943 Bengal famine, in which millions died; the slave trade, in which it is estimated at least 3.4 million black people were transported on British ships; the Irish Famine of the 1840s which killed a million – plus the numerous massacres and “native wars” in which rifle and artillery fire was turned against civilians or villagers armed with spears.

Behind these actions lay a racism which was bred into generation upon generations of Brits, rich and poor. Winston Churchill said, after the Bengal famine: “I hate Indians. They are a beastly people with a beastly religion. The famine was their own fault for breeding like rabbits.” But like Rudd, he was only echoing what his voters thought.

The colonialist mentality is still deeply rooted in British conservatism. It is the secret of their ability to treat each other, and fellow Europeans politely, but to treat people from the former colonies as troublesome, stupid, irrelevant people – whose rights are contingent on our whims or policies.

The mental gymnastics required to achieve this level of false consciousness are not hard to do. You just have to believe that Britain’s voluntary retreat from Empire after Suez makes everything it did before irrelevant, or alright, or of no modern consequence. Then you have to venerate Churchill – whose opposition to appeasing Hitler allows the mass exoneration of those who perpetrated Britain’s colonial crimes.

This attitude is what lies beneath the dual personalities of politicians like Amber Rudd and Theresa May. Polite, liberal and genteel – until they’re having you incarcerated in Yarl’s Wood, sending vans to your street telling you to “go home”, and turning you into a non-citizen of a country you’ve always lived in.

The Conservative party’s culture is pervaded with the technocratic racism that produced the Windrush scandal. The journals, magazines and websites allied to it teem with racist stereotyping – Guido Fawkes’s famous tweet of “in bed” with Diane Abbott, to Taki’s columns in the Spectator, to the now self-acknowledged Islamophobia of the Express; it is – as Sayeeda Warsi confirmed – rife with Islamophobia. As Rudd acknowledged: the real Tory attitude to Britain’s ethnic minorities is the one we saw unshackled after 2015, once the coalition ended.

The first step towards a cure is to have an open, fact based discussion about colonialism. Yet the Tory cabinet is stuffed full of people who think the British empire was a force for good. Michael Gove changed to school curriculum to force kids to learn about the benefits of white racist rule in India and Africa. Boris Johnson once described the people from the Commonwealth as “regular cheering crowds of flag-waving piccaninnies”.

Even as I write this, I know the alt-right revisionists and supporters of “race science” will react with a slew of doctored historical evidence claiming that the slave trade was a force for good, that the Irish starved themselves or that black people are genetically inferior to whites.

So the Windrush calamity is just one battle in the war for the soul of centre right parties going on all over the world. To be liberal and centrist in a period of calm and prosperity, when the far right is quarantined and xenophobic populism kept at bay is one thing. But ours is a time careering towards the extremes. Conservative parties in Spain, Austria and the USA have already moved dangerously rightwards.

Twitter, YouTube, 4chan and the comments sections of news websites reflect levels of anger and otherisation directed at non-white people, and not from bots but real, red-faced ignorant humans, that we should be working overtime to rid official politics of racism.

Windrush was not an administrative error. It was the product of a vindictive racist policy that, by the Tories’ own admission, they had been “shackled” from implementing fully during the coalition.

Corbyn – who warned about the dangers, opposed the legislation and took up the individual cases – got it right. The whole case is a reminder of the political fresh air that will blow through Westminster when he becomes prime minister.