This week, President Obama vowed to use his final two years in office to address the “simmering distrust that exists between too many police departments and too many communities of color.” The first tangible step toward that goal: the president’s request for $263 million to buy 50,000 body-worn cameras for police departments around the country, and to pay for expanded officer training. Those proposals are promising, but the president is missing a more obvious first step. He should immediately call for all law enforcement agencies to keep an accurate count of how many people the police kill each year.

As has been well-documented, the federal government keeps no such definitive tally. The closest thing to an official estimate can be found in the FBI’s annual Supplementary Homicide Report, which suggests the number is somewhere around 400 per year. But that data set is severely flawed, and the government knows it. For starters, the report counts only “justified homicides.” There isn’t even a category for those police killings that are deemed to be unjustified (on those exceedingly rare occasions that they are classified as such). More frustrating is the fact that the report is nowhere close to a full tally. The FBI relies on statistics from state and local police departments. Those same agencies, though, are under no obligation to submit the data, and most don’t. In a typical year, only about 750 of the nation’s 17,000 law enforcement agencies self-report their officer-involved shootings to the FBI.

That must change if Obama is serious about addressing the tension between the nation’s predominantly white law enforcement agencies and the minority communities they police. As the last several months have illustrated, the most high-profile source of that tension is the use of lethal force. The death of Michael Brown in Ferguson, Missouri, has garnered the most national attention, but it was hardly an isolated event. Young black males are 21 times more likely to be shot and killed by police than their white counterparts, according to a ProPublica analysis of the limited data that are available.

Without a formal and comprehensive reporting system, the president, lawmakers, and everyone else have no way of knowing the true scope of the problem. Even if the government is willing to believe that police officers are almost always justified when they kill suspects in the line of duty, Washington still owes the nation a full accounting of those killings that it has implicitly sanctioned. How can the president hope to limit the number of lives lost if he has no way of knowing how many lives are actually lost? How will Congress evaluate whether policies aimed at curbing police shootings are successful if it has no way of tracking the success or failure of those policies?

This isn’t a matter of a handful of homicides gone missing, either. The Wall Street Journal analyzed the most recent data from 105 of the nation’s largest police agencies and discovered that between 2007 and 2012 there were roughly 500 police killings that were nowhere to be found in the bureau’s national tally. The FBI’s tally suggests that there had been 1,242 justified homicides in those same departments’ jurisdictions over that period. In reality, the paper found, the number was closer to 1,800.

The WSJ investigation leads with an example of one such killing that, based solely on the federal records, never happened. In 2012, 24-year-old Albert Jermaine Payton was shot and killed in Washington, D.C., after he allegedly wielded a knife in front of police. D.C. police didn’t provide the bureau with any information on its justified homicides from that year, so Payton’s death remains uncounted by the federal government.

We don’t have to look too hard to find another example of a police killing that appears destined to go unrecognized by the feds. When the FBI releases its 2014 report, don’t expect to see Eric Garner counted. New York does not report justifiable homicide data to the FBI. The state revamped its reporting system in 2002 and again in 2006, but it still doesn’t track police killings, according to the Journal. The best a spokeswoman for the New York State Division of Criminal Justice Services could offer was that the agency was “looking to modify our technology so we can reflect these numbers.” In the meantime, the death of Garner, who was killed this summer after being put in a banned chokehold, will not be counted by the FBI.

With government agencies unable or unwilling to do this important work, news organizations and not-for-profits have attempted to fill the gap. Brian Burghart, the editor of the Reno News & Review, has spent the past two years tracking police-related deaths at his website, Fatal Encounters. Deadspin launched its own crowdsourced effort this summer, creating a database of police-involved shootings between 2011 and 2013. Perhaps the most comprehensive effort is the Gun Violence Archive, which launched last year with the goal of counting every gun-related incident in the country, including an individual tally of police-involved shootings. As of Friday, the project had counted a staggering 2,886 officer-involved shootings this year alone. That figure includes all shootings, not just deaths, but it’s still striking that it dwarfs the 420 justified homicides the FBI counted each year on average between 2009 and 2013. The more you look at these grass-roots efforts to track down the real numbers, the more absurd the “official figures” seem.

Burghart and Deadspin and the Gun Violence Archive are all doing difficult and important work, but the best they do is paint a better picture, not a complete one. They’re relying on local media reports to fill in gaps in the government data, but news stories can be outdated or incorrect, and not every death makes its way into print. “We’re not going to get everything,” says Kyle Wagner, who’s leading Deadspin’s effort. “We’d love it if we could, but that’s just not what’s going to happen.” (In 2013, Slate ran into similar problems while attempting to keep a running tally of every gun-related death in the country. After a full year our database was at least 20,000 deaths short of reality.)

It’s shameful that all of these workarounds are necessary. If Obama wants to help mend the frayed relationship between the police and the public, then he needs to make it an honest conversation. That can’t happen unless the government starts counting each and every life that law enforcement takes. Ignoring the problem won’t make it go away. The deaths of Michael Brown and Eric Garner are only the latest proof of that—and the limited data we do have make it clear they won’t be the last.