The following is an excerpt from Andrew Ferguson's 2017 book, The Rise of Big Data Policing and has been re-printed with his permission. Ferguson is a law professor at the University of the District of Columbia's David A. Clarke School of Law.

The rise of big data policing rests in part on the belief that data-­based decisions can be more objective, fair, and accurate than traditional policing.

Data is data and thus, the thinking goes, not subject to the same subjective errors as human decision making. But in truth, algorithms encode both error and bias. As David Vladeck, the former director of the Bureau of Consumer Protection at the Federal Trade Commission (who was, thus, in charge of much of the law surrounding big data consumer protection), once warned, "Algorithms may also be imperfect decisional tools. Algorithms themselves are designed by humans, leaving open the possibility that unrecognized human bias may taint the process. And algorithms are no better than the data they process, and we know that much of that data may be unreliable, outdated, or reflect bias."

Algorithmic technologies that aid law enforcement in targeting crime must compete with a host of very human questions. What data goes into the computer model? After all, the inputs determine the outputs. How much data must go into the model? The choice of sample size can alter the outcome. How do you account for cultural differences? Sometimes algorithms try to smooth out the anomalies in the data—­anomalies that can correspond with minority populations. How do you address the complexity in the data or the "noise" that results from imperfect results?

The choices made to create an algorithm can radically impact the model’s usefulness or reliability. To examine the problem of algorithmic design, imagine that police in Cincinnati, Ohio, have a problem with the Bloods gang—­a national criminal gang, originating out of Los Angeles, that signifies membership by wearing the color red.

Police use a social media scraping tool like Geofeedia to target possible gang members involved in drive-­by shootings. They search key terms like "gang," "car," "gun," "hit," "cap," "park," "run," "drive," "shoot," "strike," "red," and "colors." A series of tweets and social media posts pop up, and police begin tracking links between the various social media users.

On the basis of the identified problem (shootings), the identified targets (people using social media and making gang-­related comments), and the location (Cincinnati), the software program comes up with a list of suspects to target. But unfortunately for the accuracy of the computer program, many people in Cincinnati might be talking about the local baseball team—­the Reds—­using words like "hit," "run," "strike," "cap," "park," and other baseball terms to discuss sports. The correlation might be very accurate given the search parameters and algorithmic design—but overbroad with regard to finding the actual targets.

Similarly, the data mining of precursor crimes—­if designed poorly—­can lead to unhelpful and overgeneralized results. PredPol, which has successfully tested place-­based predictive policing models of property crimes, attempted to use a similar approach for homicides. PredPol’s starting point, rather commonsensically, was that if you traced the prevalence of handguns, you could predict future shooting deaths. Find the guns, and you will find the bodies. Using Chicago as its laboratory, PredPol compiled a data set consisting of 38,740 violent crimes during the years 2009, 2010, and 2011.

Looking at crimes with "handgun" in the descriptive field, the researchers isolated 17,020 robberies, 6,560 assaults, 8,252 weapons violations, 5,274 batteries, and 303 criminal sexual assaults. The researchers then compared those precursor handgun crimes to the 1,331 homicides during that same time frame. In a white paper published on the results, the company claimed, "PredPol successfully predicts 50 percent of gun homicides by flagging in real-­time only 10.3 percent of Chicago. . . . Crimes involving guns continue to have an impact on future gun homicides for 30–­100 days and risk spreads over as much as 1/2 mile in area."

Pulling that conclusion apart, however, demonstrates the misleading utility of the analysis. First, predicting a shooting one month to three months out does not provide much guidance to police on the streets every day. When during that three-­month time period should police be vigilant? What would they be looking for during that time? Also the area (10 percent of Chicago) is vast—­approximately 23 square miles. Again, without more granular information, this correlation, even if 100 percent accurate, does not offer enough insight to improve policing.

Algorithms can also just get it wrong. Pedro Domingos, a professor of computer science at the University of Washington and author of The Master Algorithm, recounted an important lesson of mistaken machine learning.

In an article for the Washington Post, he explained how a colleague had created a computer algorithm to tell the difference between wolves and wild dogs (which can look quite similar). During testing, the algorithm performed perfectly—too perfectly. Only upon further inspection was it revealed why the computer did so well. All of the photos of the wolves had been taken with white snow in the background, and none of the photos of the dogs had snow in the background. The computer had simply learned to identify snow, not wolves. Similar dangers face predictive algorithms; false flags might prove correlations but only because of a poorly designed model.