State of Michigan's mistake led to man filing bankruptcy

Paul Egan | Detroit Free Press

LANSING — Brian Russell never committed unemployment insurance fraud, or even attempted to do so.

And he had no idea an automated state of Michigan system had accused him of doing anything wrong until 2016, when officials seized his nearly $11,000 tax refund check.

The state finally cleared Russell in 2018, but the false fraud debacle — which has hurt tens of thousands of innocent Michigan residents — undermined his ability to provide for his two kids and led to a bankruptcy filling.

"It's devastating," Russell, a 43-year-old maintenance electrician from Zeeland, told the Free Press. "You would think if they were going to put something that huge in place, they would have someone — or even a team of people — overlooking it and making sure there were no problems."

Experts say the MIDAS (Michigan Integrated Data Automated System) false fraud fiasco, while unique to Michigan in terms of the details, is only one of the most glaring national examples of how the use of artificial intelligence by governments is harming citizens. Those most likely to be harmed by such systems, they say, are the economically disadvantaged.

"We're seeing more and more of these kinds of atrocities," said Rashida Richardson, director of policy research at the AI Now Institute, a nonprofit connected with New York University that researches the social implications of artificial intelligence.

Other examples of "intelligent" government computer systems running amok, in Michigan and elsewhere, include:

In another Michigan case, the Department of Health and Human Services used an automated system to disqualify those with outstanding felony warrants from receiving state food assistance. Between the end of 2012 and the start of 2015, the system produced false matches that improperly disqualified more than 19,000 residents from food assistance. A 2013 federal class-action lawsuit led to an out-of-court settlement and reinstatement of those improperly disqualified.

In Idaho, introduction of an automated system to determine the dollar value of disability services available to Medicaid recipients resulted in large cuts for many recipients. A court later found that the system was unlawfully arbitrary, unfair and lacked due process. There have been similar cases related to disability benefits in Arkansas and Oregon.

In Houston, where a system of algorithms was used to evaluate the performance of teachers, teachers were able to overturn the system on due process grounds. They successfully argued that because the vendor considered the evaluation system a trade secret, they were denied the right to use the data to understand or improve their performance.

In the District of Columbia, an automated system used to assess the risk for violence of youth in the juvenile justice system was found to be racially discriminatory as it was used in connection with one young defendant deemed "high risk" and in need of detention. The system is still in use.

Other concerns relate to the use of facial recognition technology, which is extensively used by police in Detroit, and "predictive policing," which the Michigan State Police has shown interest in.

The American Civil Liberties Union (ACLU) and other groups are pointing to disasters like MIDAS to push for laws that limit, regulate and increase transparency in the ways governments collect and use data for computerized decision-making.

More: Michigan residents falsely accused of jobless fraud can sue, Supreme Court says

More: State names jobless advocate to lead Unemployment Insurance Agency

Richardson said governments can be expected to continue to expand the range of applications as technology advances and the marketing of systems by software vendors expands.

The "creepiest example" of a new system Richardson is aware of is soon to be implemented in Allegheny County, Pennsylvania, where officials have been using a "family screening tool" and predictive analytics to try to head off child abuse. Starting in January, the county is planning to assign each child and family a "risk score" at birth, according to a county fact sheet and news media reports.

Jim Hendler, a computer science professor and director of the Institute for Data Exploration and Applications at Rensselaer Polytechnic Institute in Troy, New York, said many concerns about government use of artificial intelligence are well-founded and others may be overblown.

One of the major shortfalls of the MIDAS system in Michigan was insufficient testing before it was put into use, Hendler said.

When he advises governments on the appropriate uses of such systems, Hendler tells them to "keep humans in the loop to the extent possible until trust is built," he said.

Trust is important, he said.

After the Flint water crisis, scientists developed an artificial intelligence system to analyze a range of data to try to determine which homes were most likely serviced underground by lead pipes, in need of priority replacement, he said. The system had a high level of accuracy, but it was abandoned because residents had not bought into the system and were suspicious and concerned about why some homes were having their water lines replaced more quickly than others, he said. Moving away from the automated system meant that replacement of lead lines took longer than it otherwise would have, he said.

Jennifer Lord, the Royal Oak attorney representing plaintiffs in a class-action lawsuit over the MIDAS system, recently attended an international workshop in Berlin about the uses and abuses of artificial intelligence systems.

"It's really hard to wrap your head around how big this is, how fast it's moving, and how totally unregulated it is," Lord said.

"Government is outsourcing a governmental function to private entities that have no accountability and no transparency."

When lawyers or others try to find out how a system such as MIDAS could have produced so many errors, the software developers often seek to put a privacy shield over how the system is designed, citing confidential intellectual property rights, she said.

"I don't think it's too much to ask" to have transparency and public input in the development of these systems, she said.

"If we're asking a private company to carry out a government function, they should be transparent."

For Russell, who was assessed penalties multiple times higher than the alleged unemployment insurance fraud that never happened, life is finally starting to return to something near normal. Though he got back most of the money that was taken from him, with the help of the Workers' Rights Clinic at U-M Law School, far more damage was done than he ever expects to be able to recoup.

The state sent automated notices of its fraud determination to Russell's online unemployment insurance account, which he had no reason to check, because he had not been collecting unemployment insurance for years when the alleged fraud was detected.

"It ruined me," he said. And coming out of the blue, "it was like, I would almost want to say, being stabbed in the back."

Contact Paul Egan: 517-372-8660 or pegan@freepress.com. Follow him on Twitter @paulegan4. Read more on Michigan politics and sign up for our elections newsletter.