Despite survey data on defensive gun uses being notoriously unreliable, until recently there have been only scattered attempts at providing an empirical alternative. The first scientific attempt was a study in Arizona, which examined newspaper, police reports and court records for defensive gun uses in the Phoenix area over a 100 day period. At the time Arizona had the 6th highest gun death rate, an above average number of households with firearms and a permissive “shall issue” concealed carry law meaning that defensive gun use should be higher than the national average.

Extrapolating Kleck-Gertz survey results to the Phoenix area would predict 98 defensive killings or injuries and 236 defensive firings during the study period. Instead, the study found a total of 3 defensive gun uses where the gun was fired, including one instance in which a feud between two families exploded into a brawl and several of the participants began firing. These results were much more in line with (but still substantially less than) extrapolated NCVS data, which predicted 8 defensive killings or injuries and 19 firings over the same time frame.

Brand new data compiled by the Gun Violence Archive, a non-partisan organization devoted to collecting gun violence data, further confirms Hemenway’s suspicion that Kleck and Getz’s findings are absurd. The archive found that for all of 2014 there were fewer than 1,600 verified defensive guns uses, meaning a police report was filed. This total includes all outcomes and types of defensive uses with a police report—a far cry from the millions that Kleck and Getz estimated.

Many gun advocates will protest at this point that not all defensive gun uses are reported to the police, which is true. However, Kleck’s surveys and the NCVS reports indicate that more than 50 percent of such incidents are reported to the police. This would indicate 3,200 defensive uses on an annual basis, still well short of what surveys suggest. Further, if there actually are 50,000 defensive gun uses as NCVS’ data suggests, or more than 1 million as Kleck and Getz’s surveys claim, that would mean only 3.2 percent or 0.16 percent respectively of defensive gun uses are reported to the police. Believing that such a small fraction of incidents are reported is indulging in fantasy.

Kleck and Gertz often defend their paper by claiming that their results are consistent with the findings of other private surveys. They explain that the reliability of a survey should be judged by the degree to which it coheres with the estimates of other surveys. However, using a tool we know to be flawed, over and over again, does not increase the quality of estimates deriving from the tool—it merely produces convergence to an arbitrary number. Surveys, for example, regularly show that men have sex with women more often than women have sex with men. Survey results don’t mean anything if they don’t pass muster with reality.

***

The spurious conclusions in these surveys don’t just distort the pro-gun community’s perception of defensive gun use. For example, the claim that millions every year shoot their guns in self-defense has led some to posit that there are more defensive gun uses than criminal uses. This assertion is inexplicable—not backed by any substantive evidence. We have yet to find a single study examining the question that does not show that criminal uses far outweigh defensive uses.

You might hear gun advocates substantiate this claim by comparing inflated survey numbers like Kleck’s with NCVS crime numbers. But defensive gun use surveys and the NCVS use different methodologies. To compare those two data sets is to break one of the most important laws of statistical analysis: You must always compare likes to likes.

And indeed, comparing NCVS results to NCVS results yields a very different picture—that more than 9 times as many people are victimized by guns than protected by them. Respondents in two Harvard surveys had more than 3 times as many offensive gun uses against them as defensive gun uses. Another study focusing on adolescences found 13 times as many offensive gun uses. Yet another study focusing on gun use in the home found that a gun was more than 6 times more likely to be used to intimidate a family member than in a defensive capacity. The evidence is nearly unanimous.

Beyond the defensive gun use versus criminal use dichotomy lies an important question: Are all defensive gun uses good? Undergirding gun advocates’ rhetoric touting the millions of defensive gun uses every year is the assumption that these uses are necessarily good. However, most cases of defensive gun use are not of gun owners heroically defending their families from criminals.

Kleck himself admitted in 1997, in response to criticism of his survey, that 36 to 64 percent of the defensive gun uses reported in the survey were likely illegal—meaning the firearm was used to intimidate or harm another person rather than for legitimate self-defense. His conjecture was confirmed by a Harvard study showing that 51 percent of defensive gun uses in a large survey were illegal according to a panel of 5 judges. This was even after the judges were told to take the respondents at their word, deliberately ignoring the tendency of respondents to portray themselves in a positive light.

Let’s assume for a moment that Kleck and Getz’s estimates are accurate. Rather than being a boon to civilized society, then, these estimates of 1 million to 2.5 million defensive gun uses annually would instead indicate an epidemic of irresponsible gun owners—millions! Lucky for us, despite what the NRA’s favorite criminologists claim, this clearly isn’t the case.

The myth of widespread defensive gun use is at the heart of the push to weaken already near catatonic laws controlling the use of guns and expand where good guys can carry guns to bars, houses of worship and college campuses—all in the mistaken belief that more “good guys with guns” will help stop the “bad guys.” As Wayne LaPierre of the NRA railed in the wake of the Sandy Hook shooting: “The only thing that stops a bad guy with a gun, is a good guy with a gun.”

But the evidence clearly shows that our lax gun laws and increased gun ownership, spurred on by this myth, do not help “good guys with guns” defend themselves, their families or our society. Instead, they are aiding and abetting criminals by providing them with more guns, with 200,000 already stolen on an annual basis. And more guns means more homicides. More suicides. More dead men, women and children. Not fewer.