William Gibson wrote that the future is here, just not evenly distributed. The phrase is usually used to point out how the rich have more access to technology, but what happens when the poor are disproportionately subject to it?

In Automating Inequality, author Virginia Eubanks argues that the poor are the testing ground for new technology that increases inequality. The book, out this week, starts with a history of American poorhouses, which dotted the landscape starting in the 1660s and were around into the 20th century. From there, Eubanks catalogues how the poor have been treated over the last hundred years, before coming to today’s system of social services that increasingly relies on algorithms.

Eubanks leaves no uncertainty as to her position on whether such automation is a good thing. Her thesis is that the punitive and moralistic view of poverty that built the poorhouses never left us, and has been wrapped into today’s automated and predictive decision-making tools. These algorithms can make it harder for people to get services while forcing them to deal with an invasive process of personal data collection. As examples, she profiles three different programs: a Medicaid application process in Indiana, homeless services in Los Angeles, and child protective services in Pittsburgh.

Eubanks spoke to MIT Technology Review about when social services first became automated, her own experience with predictive algorithms, and how these flawed tools give her hope that inequality will be put into such stark relief that we will have to address how we treat our poor, once and for all.

What are the parallels between the poorhouses of the past and what you call today’s digital poorhouses?

These high-tech tools we're seeing—I call it “the regime of data analytics”—are actually more evolution than revolution. They fit pretty well within the history of poverty policy in the United States.

When I originally started this work, I thought the moment that we’d see these digital tools really arrive in public assistance and public services might be in the 1980s, when there was a widespread uptake of personal computers, or in the 1990s when welfare reform passed. But in fact, they arose in the late 1960s and early 1970s, just as a national welfare rights movement was opening up access to public assistance.

At the same time, there was a backlash against the civil rights movement going on, and a recession. So these elected officials, bureaucrats, and administrators were in this position where the middle-class public was pushing back against the expansion of public assistance. But they could no longer use their go-to strategy of excluding people from the rolls for largely discriminatory reasons. That's the moment that we see these technologies arrive. What you see is an incredibly rapid decline in the welfare rolls right after they’re integrated into the systems. And that collapse has continued basically until today.

So for some of these algorithms that we have right now, machine-learning tools will replace them. In your research did you come across any issues that are going to arise once we have more AI within these systems?

I don’t know that I have a direct response to it. But one thing I will say is that the Pittsburgh child services system often gets written about as if it’s AI or machine learning. And in fact, it’s actually just a simple statistical regression model.

I do think it’s really interesting, the way we tend to math-wash these systems, that we have a tendency to think they're more complicated and harder to understand than they actually are. I suspect that there's a little bit of technological hocus-pocus that happens when these systems come online and people often feel like they don't understand them well enough to comment on them. But it’s just not true. I think a lot more people that are currently talking about these issues are able to, confident to, and should be at the table when we talk about them.

You have a great quote from a woman on food stamps who tells you her caseworker looks at her purchase history. You appear surprised, so she says, "You should pay attention to what happens to us. You're next." Do you have examples of technologies that the general population deals with that are like this example?

I start the book by talking about a case where my partner was attacked and very badly beaten. After he had gotten some major surgery, we were told at the pharmacy when I was trying to pick up his pain meds that we no longer had health insurance. In a panic, I called my insurance company and they told me basically that we were missing a start date for our coverage.

I said, “You know, well, that’s odd, because you paid claims that we made a couple of weeks ago, so we must have had a start date at that point.” And they said, “Oh, it must have just been a technical error. Somebody must have accidentally erased your start date or something.”