Algorithm blues

The promise and peril of big-data justice

Can algorithms accurately predict where crime will occur?

EIGHT storeys above downtown Los Angeles, Sean Malinowski, deputy chief of the Los Angeles Police Department (LAPD), focuses intently on a computer map of his old stomping ground. Nestled between Burbank and Santa Clarita, the Foothill district is a hotch-potch of industrial and residential districts riven by highways. Mr Malinowski ran its police station before his promotion moved him downtown.

Colourful dots representing reported crimes freckle the map like psychedelic pimples. Adjacent to some of the dots are red squares. Each one represents a 250,000-square-foot (2.3-hectare) area that PredPol, a crime-prediction software used by the LAPD and at least 50 other law-enforcement agencies around the world, has flagged as being at risk of future criminal activity. Mr Malinowski says that, if he were still in charge of policing in Foothill, he would ask his officers to drive through those areas frequently, “so we’re there randomly—it throws the criminals off.” The idea is not to nab people red-handed, but to deter them through increased police presence.

PredPol is just one of a number of firms offering crime-prediction software to police forces. While the precise components of each firm’s algorithms probably differ, the broad idea is the same. They aim to help police allocate resources efficiently by using large amounts of data to predict (and therefore prevent) crime.

The use of algorithms to tackle complex problems such as urban crime, or to try to forecast whether someone is likely to commit another crime, is not inherently alarming. An algorithm, after all, is just a set of rules designed to produce a result. Criminal justice algorithms organise and sort through reams of data faster and more efficiently than people can. But fears abound: that they remove decisions from humans and hand them to machines; that they function without transparency because their creators will not reveal their precise composition; that they punish people for potential, not actual, crimes; and that they entrench racial bias.

Defenders of such programmes argue, correctly, that police have always relied on prediction in some form. Officers line parade routes, for instance, because experience has shown that the combination of crowds, alcohol and high spirits create an increased public-safety risk. Eliminating prediction from policing would produce an entirely reactive force. All these programs do, defenders say, is harness more data from more sources to help police make better decisions.

But the algorithms on which police base their decisions are, as far as the public is concerned, black boxes. The companies that create and market them consider their precise composition trade secrets. “Algorithms only do what we tell them to do,” says Phillip Atiba Goff of John Jay College of Criminal Justice in Manhattan. If their creators feed them biased data they will produce results infected with bias. And predictive policing is just one way in which the criminal-justice system is using algorithms to help them make decisions.

New Jersey uses an algorithm based on past criminal history, age, past failure to appear at trial and the violence of the current offence to determine whether someone is suitable for bail—that is, whether he presents too great a risk of flight or of committing more crimes while awaiting trial. Several states use algorithms to provide sentencing recommendations. At least 13 American cities use them to identify people likely to become perpetrators or victims of gun violence.

NYPD, too

The first time such approaches came to public notice was in the 1990s, when William Bratton introduced CompStat, a statistically driven management system, into the New York Police Department (NYPD), which he ran. CompStat involved regular meetings of commanding officers discussing prevention strategies and recent crime data from their precincts. As one former NYPD deputy commissioner says, CompStat encouraged police to ask, “What is the problem? What is the plan? What are the results to date?” and to use data to answer all of those questions.

But CompStat was largely reactive rather than predictive. It also used precinct-wide data, while software such as PredPol can target enforcement to specific blocks. Crime does not occur randomly across cities; it tends to cluster. In Seattle, for instance, police found that half of the city’s crime over a 14-year period occurred on less than 5% of the city’s streets. The red squares in Foothill cluster around streets near junctions to main roads—the better to burgle and run while homeowners are at work—as well as around businesses with car parks (lots of inventory, empty at night) and railway stations. Burglars who hit one house on a quiet street often return the next day to hit another, hence the red squares.

And, unlike CompStat, which used arrests as a measure of officers’ productivity, PredPol aims to prevent rather than punish crimes. “I’m more concerned about the absence of crime” than citations and arrests, says Mr Malinowski. “We don’t want mass incarceration for little crimes.” As for measuring productivity, that, too, has grown easier. LAPD patrol cars are geotagged, and the red boxes geofenced, so senior officers know precisely how long each car spends there.

Exactly what data get fed into the algorithms varies by company. Some use “risk-terrain modelling” (RTM), which tries to quantify what makes some areas crime-prone. One RTM algorithm uses five factors: prevalence of past burglaries, the residence of people arrested for past property crimes, proximity to main roads, geographic concentration of young men, and the location of apartment buildings and hotels. Some include requests for police help, weather patterns and the proximity of bars or transport stations. PredPol uses reported, serious crimes such as murder, aggravated assault and various forms of theft, as well as the crime’s date, time and location. Most of these algorithms use machine learning, so they are designed to grow more accurate the more predictions they make and the more data they take in.

Some analytic programmes suck in and link up more data. A joint venture between Microsoft and the NYPD called Domain Awareness System pulls data from the city’s thousands of publicly owned CCTV cameras, hundreds of fixed and car-mounted ANPRs, and other data sources. The NYPD says its system can track where a car associated with a suspect has been for months past, and can immediately alert police to any criminal history linked with a flagged number plate.

You have the right to remain silent

So do these algorithms work? Do they accurately forecast where crime will occur and who will go on to commit future crimes? Here the evidence is ambiguous. PredPol touts its 21-month-long trials in Kent, an English county, and Los Angeles, which found that the programme predicted and helped to prevent some types of crime (such as burglary and car theft) more accurately than human analysts did. A trial in Louisiana of a different data-driven predictive-policing model, however, found no statistically significant reduction in property crimes compared with control districts.

But even if such approaches proved effective beyond a doubt, concerns over their potential to trample civil liberties and replicate racial bias would remain. These concerns are most acute for algorithms that implicate people rather than places. The Chicago police department has compiled a “strategic subject list” of people it deems likely to be perpetrators or victims of gun violence (both groups tend to comprise young African-Americans from the city’s south and west sides). Its central insight parallels that of geographic predictions: a small number of people are responsible for a large share of violent crime. The department touts its accuracy. In the first half of 2016, it says, 74% of gun-violence victims and 80% of those arrested for gun violence were on the list.

Police say they update the list frequently. When someone new shows up on it, officers will sometimes visit that person’s home, thus promoting contact with police before a person has committed a crime. Nobody knows precisely how you end up on the list, nor is it clear how (short of being shot dead) you can get off it. One 22-year-old man, Robert McDaniel, told the Chicago Tribune that police came to his home and told him to straighten up—even though he had just a single misdemeanour conviction (he may have been earmarked because a childhood friend with whom he was once arrested was shot dead).

In a study of the first version of the list from 2013, RAND, a think-tank, found that people on it were no more likely to be victims of a shooting than those in a random control group. Police say the current list is far more accurate, but have still refused to reveal the algorithmic components behind it. And both Chicago’s murder rate and its total number of homicides are higher today than they were when police started using the list in 2013.

Meanwhile, algorithms used in sentencing have faced criticism for racial bias. ProPublica, an investigative-journalism NGO, studied risk scores assigned to 7,000 people over two years in Broward County, Florida, and found black defendants twice as likely as whites to be falsely labelled at high risk of committing future crimes. It also found the questions predicted violence poorly: only around 20% of those forecast to commit violent crimes actually did so. Northpointe, the firm behind the algorithm, disputed ProPublica’s findings.

But the questions on Northpointe’s risk-assessment form illustrate how racial bias can infect an algorithm even without any direct questions about race. It asked how often a defendant, his family members and friends have been arrested. Those numbers will presumably be higher in poor, overpoliced, non-white districts than rich ones. It also asked whether friends were in gangs, how often the defendant has “barely enough money to get by” and whether it is “easy to get drugs in your neighbourhood”—all questions that ethnic minority defendants will, on average, answer affirmatively more often than white ones. More broadly, a proprietary algorithm that recommends a judge punish two people differently based on what they might do offends a traditional sense of justice, which demands that punishment fit the crime not the potential crime.

Another analytical system, called Beware, assigns “threat scores” in real time to addresses as police respond to calls. It uses commercial and publicly available data, and it has a feature called Beware Nearby, which generates information about potential threats to police near a specific address, meaning officers can assess the risk when a neighbour calls the emergency services.

This raises privacy concerns, but it could cause other problems, too. For instance a veteran who has visited a doctor and taken medicine prescribed for PTSD, who also receives gun catalogues in the post, could be deemed high risk. Police might then approach his house with guns drawn, and it is not hard to imagine that kind of encounter ending badly. Such threat scores also risk infection with bad data. If they use social-media postings, they also raise free-expression concerns. Will police treat people differently because of their political opinions?

Questions of bias also surround place-based policing. Using arrests or drug convictions will almost certainly produce racially biased results. Arrests reflect police presence more than crime. Using drug convictions is suspect, too. Black and white Americans use marijuana at roughly similar rates, with the rate for 18- to 25-year-olds higher for whites than blacks. But blacks are arrested for marijuana possession at nearly three times the rate of whites across America—and even more often than that in some districts. Black people in Washington, DC, and Iowa are eight times likelier than whites to face arrest for marijuana. Charges for possession of that one drug comprise half of all drug arrests. Small wonder that a study by Kristian Lum of the Human Rights Data Analysis Group and William Isaac found that when a predictive algorithm was trained on historical drug-crime data in Oakland, California, it targeted black areas at twice the rate of white ones, and low-income neighbourhoods at twice the rate of high-income ones.

Place-based prediction also raises questions about reasonable suspicion. If police are on a residential block algorithmically predicted to be at risk of theft, and they drive past a man carrying a heavy satchel, does that justify stopping and searching him, especially when they might not do the same on another block?

Some accept that algorithms may replicate racial biases, but say they at least do not aggravate them. “It’s not a perfect world,” says one advocate of algorithm-based bail reform. You need to compare risk-based assessments with the status quo, he says. If a black and a white defendant came before a judge with the exact same record today, the judge might treat the black defendant worse. “At least with the risk assessment they’ll get the same score.” But that is a depressingly low bar to set.