The first man to be arrested in Chicago based on facial recognition analysis was sentenced last week to 22 years in prison for armed robbery. The Chicago Sun-Times reported that the Chicago Police Department acquired the technology via a $5.4 million federal grant.

In February 2013, Pierre Martin robbed a man at gunpoint while on a Chicago Transit Authority (CTA) train. After taking the man’s phone, Martin jumped off the train. However, his image was captured by CTA surveillance cameras and was then compared to the Chicago Police Department’s database of 4.5 million criminal booking images. Martin, who already had priors, had a mugshot in the database. He was later positively identified by witnesses.

At trial, Martin also admitted to committing a similar robbery also on the Pink Line in January 2013—his face was captured during both robberies.

Variables matter

Facial recognition is becoming an increasingly common tool in law enforcement, used by organizations ranging from the National Security Agency down to local police. While the technology is improving, it’s had limited success in some high-profile situations, such as the manhunt for the Boston bombing suspect in 2013.

Some cops have even used scans of images on driver’s licenses to find fake ones: earlier this year, the New Jersey Attorney General's Office charged 69 people with obtaining false driver's licenses after running facial recognition analysis. In 2013, the New York Governor’s Office announced that more than 2,500 people on identity theft charges had been arrested as the result of similar scans.

Anil Jain, a biometrics expert and professor of computer science at Michigan State University, told Ars that getting a match depends greatly on the quality of the image, which may be low resolution or taken from a video still. “If the face in video is close to frontal and the image quality is good, matching accuracy is very high,” he said. “We can further filter the large mugshot database by specifying the gender, race, and age group of the suspect. This filtering step also improves the matching success.”

Jain added that he and some colleagues published a research paper (PDF) in March 2014 dealing with this exact problem.

Questions linger

Elizabeth Joh, a criminal law professor at the University of California Davis, told Ars that she was not aware of a police department “that claims to have caught a suspect solely through facial technology.”

“The greatest concern with any big data tool, whether facial recognition technology or [license plate readers], is its accuracy and the amount of error we should tolerate when the police decide to interfere with individual liberty,” she said.

Meanwhile, Jay Stanley, a policy analyst with the American Civil Liberties Union, said that his organization is not opposed to the use of the technology when used in a narrow way. “The question that needs to be asked is: for every success, how many other situations are there?” he said. “How many times has this technology been used and not resulted in a prosecution? How many people have been scrutinized by the police when their face came up in a false match?”

Neither the Cook County State’s Attorney’s Office nor the Chicago Police Department immediately responded to requests for comment.