A sea change is under way in governments worldwide, and no one is talking about it. Algorithms and artificial intelligence are taking over welfare payments systems, and with this shift to automation the last vestige of dignity for poor and vulnerable people is being attacked.

These changes are often made quietly, with little or no public debate or accountability. Yet in the process, the lives of millions of people across the globe are being affected.

Last month, the Guardian ran a series, Automating Poverty, that sought to cast a light on the way digital innovation is threatening the poor. The series came about through our reporting on the work of Philip Alston, the UN rapporteur on extreme poverty and human rights.

We first came in contact with Alston when he was conducting a UN audit into extreme poverty in the US. His report was so excoriating of the Trump administration and its tax cuts that he provoked a furious reaction from the US state department, which we also covered.

We then followed Alston as he researched and wrote an equally devastating account of austerity in the UK – with a similarly splenetic response from Westminster.

In the wake of those groundbreaking and headline-creating studies, Alston decided that his next big venture would be into a subject both more global and more insidious: the emergence of what he calls the “digital welfare state”. The development has huge implications for low-income and vulnerable people, yet virtually nobody appears to be paying attention to it.

When the Guardian learned that Alston was thinking along these lines, our ears pricked up. This seemed a natural fit, not only given our emphasis on reporting on poverty but also taking into account our conviction that major social change should always occur with full transparency, not in the dark, as seemed to be the case here.

We also had the reporting muscle in place to take on a global phenomenon. With major offices in the UK, US and Australia, and correspondents located around the world, we saw the opportunity to turn our spotlight on how the digital welfare state is unfolding in both industrialised and developing countries.

Our correspondent Rebecca Ratcliffe travelled almost 1,000 miles from New Delhi to Dumka in the east of India to investigate the country’s vast biometric ID scheme, Aadhaar. When she reached the village, she was immediately led by Alabati Devi to a patch of land where, in May, her husband, Motka Manjhi, had collapsed and died.

Philip Alston, the UN rapporteur on extreme poverty and human rights. Photograph: Christopher Thomond/The Guardian

Devi is convinced Manjhi died from starvation after his food subsidies were stopped because his thumbprint wasn’t recognised by the Aadhaar biometrics database.

“The Aadhaar system is almost completely opaque,” Ratcliffe told us after she had returned to New Delhi. “In Jharkhand, one of India’s poorest states, vast numbers are affected, yet when we were reporting this story, no government officials returned our calls.”

In Australia, our Melbourne-based reporter Luke Henriques-Gomes has been covering inequality and welfare for the past year. After reporting on families who have been informed by text message that their payments have been suspended, sometimes in error and with no human to complain to, he was shocked by how little scrutiny those changes have received.

Even though the controversy around automated welfare debt has been bubbling for years in Australia, the government has managed to use similar technologies to cut the welfare budget with very little attention: in just 12 months, welfare payments were stopped an extra 1m times.

“What became clear after talking to people affected by this is how hard these automated systems are to navigate, especially for the vulnerable,” Henriques-Gomes says. “Welfare recipients were completely resigned to the fact these advancements in technology were designed to police their lives and punish them. It’s heartbreaking.”

In the UK, Robert Booth and Sarah Marsh spent the best part of six weeks digging into the automation of welfare systems. The early days were hard going: the terrain was murky, complicated and riddled with IT jargon. Most of the people who knew what was really going on didn’t want to talk. The Department of Work and Pensions declined to answer freedom of information requests, while councils such as Sunderland, which had abandoned a multimillion-pound algorithmic-welfare project, clammed up.

What kept the reporters going amid these hurdles were the experiences of claimants. “Watching people with learning difficulties being moved to tears of frustration by the vagaries of the ‘digital by default’ system was galvanising,” Booth says. “The more people we spoke to in Rochdale, London, Gateshead and Manchester, the clearer the problems with the existing systems seemed.”

A woman scanning her fingerprints for use in India’s Aadhaar biometric ID scheme. Photograph: Saumya Khandelwal/Reuters

As they dug further into the story, clear themes started to emerge about what authorities were up to. Better data and more candid responses began to come in and the reporters’ understanding grew. “Finally it felt like we had a toehold on the story, albeit with a sense that there is so much more still to find out,” Booth says.

That comment chimed with the sentiment shared by everyone who worked on the Automating Poverty project: we had alighted on a line of reporting that has only just begun.

That sense was also expressed in the scores of emails that we received in the wake of publishing the series, from social scientists, government officials – and, crucially, welfare recipients – from all around the world.

At the end of the week that the series ran, Alston’s report was published and presented at the United Nations general assembly in New York. Marking the occasion, the Guardian mediated a panel discussion to debate the human rights challenges of the digital age. The panel consisted of Alston; Michelle Bachelet, the UN high commissioner for human rights; Chris Hughes, a co-founder of Facebook; Kumi Naidoo, the secretary general of Amnesty International; and the writer and scholar Shoshana Zuboff.

Bachelet summed up the feeling that the world had glimpsed a radical shift that needed to be dragged into the light. “The dark end of the digital spectrum threatens not just privacy and safety, but undermines free and fair elections; jeopardises freedom of expression, information, thought and belief; and buries the truth under fake news. The stakes could not be higher,” she said.

“We must ensure that the digital revolution is driven by people, not the other way around.”



