Being inhuman, the former Tory policy guru Steve Hilton once wrote, is “not the natural order of things”. People rarely set out to be callous and unfeeling, or to make others’ lives unnecessarily miserable. But, as he argued in his book More Human, the way the world is structured, and the pressures placed on perfectly reasonable individuals, make them lose sight of other people’s humanity. How ironic that the party he once advised has become such a textbook illustration of what he meant.

Are Home Office civil servants particularly anti-migrant? | David Walker Read more

The common thread in so many of the crises now engulfing Theresa May’s government is policy that completely fails to recognise the complexity of people’s lives: a sort of rigid, soulless, unthinking bureaucracy that leads to casual cruelty. It’s most obvious in the Windrush scandal, where even the current home secretary admitted her department had become so obsessed with delivering immigration policy that it lost sight of the individuals behind the case numbers.

But it’s there in the cabinet minister Esther McVey’s argument that women who get pregnant as a result of rape might really benefit from having to recount their ordeal to a total stranger, in order to access benefits. And it’s there too in the countless avoidable traps sprung by universal credit. This week alone, MPs heard of women who have fled domestic violence choosing to go back because long delays in accessing the benefit left them no other means of support, while the SNP’s Philippa Whitford raised the issue of violent partners exploiting universal credit to control and trap women. (Child benefit was paid to mothers, but under the new system the money can go to either parent, and a controlling man will naturally demand he gets it.) The government’s answer is that women can ask for the payment to be split but how many would dare, given the reaction when he finds out? Heads would roll if comfortable middle-class families, encountering nothing worse from officialdom than over-zealous parking tickets, were treated like this.

Talk of “faceless” bureaucracy ignores the fact that bureaucracy has a million faces, some doubtless as depressed as anyone would be about spending their days rejecting appeals or dishing out sanctions. Border officials say they used to be allowed to exercise discretion: that given a Windrush case they could have asked a few questions about life in Britain, used their experience to judge whether the person really had been here for decades, and stamped passports accordingly. But that changed in 2011 after a furious row between Theresa May and Brodie Clark, the then head of the UK border force, deemed to have exceeded his authority by relaxing border checks to beat airport queues. Cases referred higher up might still reach someone capable of exercising compassion, but when everyone is under pressure to meet immigration targets or cut welfare budgets, the incentive is not to refer. Eventually staff get jaded, feelings are switched off, and MPs handling immigration cases describe a climate where “no” is the reflex answer.

There is nothing wrong with government tackling illegal immigration (the clue is the word “illegal”). These aren’t always victimless crimes, especially where trafficking people into exploitation is involved. But in practice, creating what May infamously called a “hostile environment” for illegal immigrants is almost impossible without that hostility affecting legal migrants and British-born people of colour. Threatening landlords, employers, NHS staff and universities with fines if they don’t check the papers of anyone seeking their services has created a reluctant army of amateur border guards incentivised to be suspicious of anyone with an accent, a foreign-sounding name, a darker skin. It didn’t take the Windrush affair to tell us how that was working out.

Play Video 2:15 Theresa May apologises for treatment of Windrush generation – video

Half of landlords are now reluctant to rent to foreign nationals, according to a survey by the Joint Council for the Welfare of Immigrants, while Britons of colour who happened not to own passports were 26% more likely than white Britons without passports to be rejected when asking about a flat to let. Crucially, landlords didn’t discriminate between black and white renters with passports, suggesting this isn’t straightforward racism.

Meanwhile some migrant parents are now so afraid of officialdom they’ve stopped claiming free school meals or even attending parents’ evenings, the Teach First executive (and former Tory special adviser) Sam Freedman tweeted this week. After the Grenfell fire, there were reports of survivors not wanting to be checked over in hospital because they feared having their immigration status questioned. The haunting question is whether some people who are here perfectly legally are now wondering if it’s safer just to disappear off the radar than risk wrongful deportation.

Doubtless May didn’t intend that, any more than Iain Duncan Smith meant to push women back into violent relationships. But they were warned, and that’s how bad things happen under governments of all political persuasions: not through some cartoonish idea of evil but through flawed ideas – doggedly pursued for ideological reasons, or because they’re popular, or because all the criticism seems to be coming from the “usual suspects” – unravelling on contact with reality. And when things start going wrong, a defensive bureaucracy reacts painfully slowly. Nobody meant to be inhuman, yet here we are.

Avoiding such misery in the future will take changes in the way parties govern, not just a change of governing party. It will mean ensuring policy is made by a diverse Whitehall workforce, for a start; one wonders if a roomful of civil servants descended from the Windrush generation would have been relaxed about destroying landing cards. But it also means politicians in all parties grasping that incremental change has a lot going for it: it’s not weak but wise to consult widely, pilot new ideas carefully and heed parliament’s early-warning systems before doing things that risk blowing up in all our faces.

Beware too any one-size-fits-all policy – flat taxes, nationwide benefit caps, universal credits or universal basic incomes – that depends on sweeping away lots of pernickety-sounding exemptions, because all too often it is loopholes and compromises that make policies human. Ideas can be simple, but people are infinitely complicated. Cruelty creeps in where we forget that.

• Gaby Hinsliff is a Guardian columnist