As the FBI follows The Counted with an effort to track people killed by officers – one dependent on voluntary law enforcement compliance – the Bureau of Justice Statistics has quietly launched a count it hopes will be comprehensive

US government database hopes to tell 'whole story' of police killings after year of Guardian count

A burglary suspect fleeing police dogs and a helicopter in Florida wades into a dark lake and disappears. Three weeks later his remains are found inside an alligator.

Was he killed by police?

It is an extreme example of the difficulty faced with increasing frequency by data scientists working on a new US government count of deaths in interactions with police – a count that appears likely to soar beyond all previous attempts, now that the issue has reached the highest levels of both protest and power.

As esoteric as the task may seem, the objective is deadly serious: to measure the true dimensions of an epidemic of lethal violence committed by police across the country on often unarmed civilians. A majority of the victims, such as Chicago teenager Laquan McDonald, die in police gunfire. Others, such as the New York father Eric Garner, may die in a banned chokehold or, like the Baltimore 25-year-old Freddie Gray whose death is currently being prosecuted, from injuries in a police vehicle.

FBI effort to track police killings is key step toward leaving no case uncounted Read more

At the start of 2015, the Guardian launched The Counted, a public-service project tallying and shedding light on such cases, which has reached a tally of 1,068 so far. Last week, the Federal Bureau of Investigation revealed plans for a similar counting effort, after grossly misrepresenting the problem in eight previous years with annual figures averaging 423.

Quietly watching both efforts has been a team of statisticians and criminologists working in a complex of low buildings in an office park housing IBM outside Raleigh, North Carolina. They are employees of RTI International, a research company with significant federal contracts from the Department of Education and the Pentagon.

The number crunchers’ latest contract is one in high demand – and a long time coming: a major deal with the Department of Justice to design a single official count that so far has eluded the government, despite swelling national demand, of people who die in interactions with police.

The DoJ’s new effort is the result of pressure from the top of the Obama administration, from the president to the attorney general, Loretta Lynch, who called in October for “national, consistent” data on people killed by law enforcement.

Pressure has also mounted from public embarrassment inside the FBI and elsewhere in the Justice Department that counts by the media this year have surpassed any accounting the government has to offer for itself. At a recent summit, the FBI director, James Comey, called it “unacceptable” that a Guardian US investigative unit had a better tally than his agency’s near 35,000 employees.

There is abundant doubt among top criminologists, however, about the ability of the FBI to do the job, despite Comey’s personal focus on the issue, because the bureau relies on the voluntary compliance of the country’s 18,000 law enforcement agencies to report to the government when someone dies in connection with police action. And the number crunchers say police overwhelmingly will not volunteer that information.

“I’m not sure what the participation rate’s going to be, if the FBI’s using – simply by encouragement,” said Janet Lauritsen, a criminologist at the University of Missouri who is chairperson of a National Academy of Sciences panel on modernizing crime statistics. “I think they’re really going to need some incentives here, and pressure – as well as pressure from the public.”

Facebook Twitter Pinterest Duren Banks, right, with the data scientist Paul Ruddle. Photograph: Roger May/The Guardian

With that shadow of doubt overhanging the FBI’s effort, a much smaller and more specialized agency within the Justice Department, the Bureau of Justice Statistics (BJS), has stepped up to propose its own way forward – and in effect take over from media efforts following the call from Obama’s cabinet.

Last summer, to little public attention, the team at RTI, working on a contract with BJS, launched a pilot program to count what criminologists refer to as arrest-related deaths. It would entail a team of about two dozen analysts working part-time to identify potential cases and then verify them with law enforcement agencies and medical examiner’s offices.

The crux of the pilot program are two computer servers dedicated to combing the internet 24 hours a day for reports of anything that looks like someone dying in an interaction with a police officer.

The humming hardware is a gamble in the sense that government statisticians had never before fully committed to using open-source techniques in their efforts to quantify fatal police use of force.A previous BJS effort to count arrest-related deaths had experimented with open-source counting.

But in a year in which the Guardian’s use of open sources – and, increasingly, reader tips – has produced a portrait of a problem orders of magnitude larger than previously acknowledged, and with pressure also growing from Congress, the Justice Department statisticians decided to jump with both feet into online sourcing.

Freddie Gray – would he have been excluded from the count? If it had just been shootings, then obviously, yes. Duren Banks, RTI statistician

The statisticians’ new effort is not limited to fatal shootings, but aims to capture the entire corpus of cases of people who die in police violence.

“Limiting it to certain types of death doesn’t tell the whole story, and so we wanted to know more about all individuals who died in police custody,” said Duren Banks, the RTI statistician and criminologist who is running the project. “With Freddie Gray, for example, would he have been excluded from the count? If it had just been shootings, then obviously, yes.”

For the pilot program, which expects to deliver an initial report to the Justice Department early next year, RTI set out to capture any death that happened while the decedent’s “ability to leave” was restricted by a law enforcement officer (excluding federal officers) – but before the decedent was officially booked into jail.

Counting police killings in the US: landmark stories that led to change Read more

That would include, significantly, anyone shot dead by police. But it would also include a case like that of Gray or Garner. It would include a person who committed suicide in a standoff with police, or a drunk driver killed in a single-car accident after pursuit by police. Ultimately, the burglar killed by the alligator would make the list, too, as someone whose freedom to leave was restricted by police at the time he was killed.

After a potential case is identified, the government-funded liaisons contact law enforcement agencies to ask them to fill out a 19-point questionnaire about each incident. The analysts also review the channels the information had taken to surface for their appraisal.

“For each case that we identify, we have recorded how we identified those cases,” said Michael Planty, the BJS official overseeing the project. “Whether it’s through, say, the Guardian, but also through open sources. So we might go to your website and find a case, but also through Google alerts we get the same information. Just understanding that is part of our assessment.”

How the BJS pilot program to count arrest-related deaths works:

Computer programs identify potential arrest-related deaths and add them to a queue for analysts.

Analysts move through the queue item by item, labeling cases “yes”, “no” or “unknown”.

For all “unknown” or “yes” cases, analysts collect information including the decedent’s name, age, date of death and the law enforcement agency involved.

Each month, analysts aggregate cases into a “unique decedent list”, eliminating duplicates.

Analysts check these lists against The Counted and other databases of people killed by police. Cases move to liaisons.

Liaisons contact the relevant law enforcement agency and/or medical examiner’s office and ask officials to fill out a 19-point questionnaire about the incident.

Liaisons contact an additional “stratified random sample” of law enforcement agencies of various sizes that were not associated with any report of an arrest-related death, to see whether any of them have in fact had such an incident.

RTI is preparing a first report to submit to BJS this spring.

The open-source mining is particularly muscular, explained RTI data scientist Paul Ruddle, thanks to the use of multiple commercial media monitoring services to get a depth of view not available through free search engines such as Google.

When the project was under design, Ruddle led a team that put together a list of dozens of keywords that would catch even obscure cases for review in deep web searches.

“We couldn’t have done this 30 years ago,” Ruddle said. “Now, because we have all this information available online in a machine-readable context, we are able to narrow down the universe of articles or posts initially without having a person read the universe of articles.”

The resulting volume of potential cases, Banks said, is substantial but manageable by a handful of rotating reviewers working to decide whether to promote potential cases to likely cases warranting follow-up.

How many people die each year in interactions with police? RTI is not disclosing its initial results, not having completed its first round of interviews with law enforcement agencies. But Banks said the number of cases it had tentatively identified so far was roughly in line with the Guardian’s tally, which she praised as exemplary “compared to the other ones that are out there beside BJS’s”.



The RTI pilot program was designed by “taking stock of what’s there and not moving ahead too quickly”, she said.

“Also looking into the field – looking at what you all were doing, looking at what some law enforcement agencies were doing to capture this information, and just trying to harness again what’s best about working within each of those approaches.

“I think that there’s certainly a thinking and an expectation, and I share this, that we should be able to do this,” Banks said. “That we should be able to know how many people die in law enforcement custody.

“It’s really important to BJS that they get 100%.”