UPDATED, originally Posted April 3, 2018: An article at Vox has gained attention for illustrating America’s “unique gun violence problem” in 17 maps and charts. A similar New York Times opinion piece by Nicolas Kristof on “How to Reduce Shootings” and another at the BBC titled “America’s gun culture in 10 charts” have gained attention. Below, we will respond to the individual graphs on Vox’s and then on Kristof’s terms. But there are a lot of assumptions behind their graphs. Often, just one or two public health studies are cited to make a particular point, without discussing any of the known weaknesses with these studies. It would also be valuable for both to acknowledge and critique research that reaches the opposite conclusions.

Economists focus on the notion of substitution. If you take away guns, people might commit suicides or murders in other ways. It is total deaths that need to concern everyone. All these pieces focus exclusively on gun deaths without looking at the larger picture.

There are many countries that have higher gun homicide rates than the United States, but simply don’t report firearm homicide data. Many of these meet the criteria to be members of the OECD (Organization for Economic Co-operation and Development). While 192 countries report total homicides, only 116 countries report firearm homicides. The average homicide rate among countries that don’t have firearm homicide data is 11.1 per 100,000. The US rank based on firearm homicides looks higher than it actually is because the high homicide countries don’t report their firearm homicide rates.

Homicide is not synonymous with murder. Homicides count both murders and justifiable homicides (when a police officer or a civilian kills someone in self-defense). In the five years from 2011 to 2015, the US experienced 11,577 firearm homicides and 8,786.4 firearm murders. This gap is much larger in the US than in other countries, so comparing homicide rates gives a more unfavorable impression of the US than if we looked only at murder rates.

Murder isn’t a nationwide problem in the United States; there are vast swathes of the country that don’t experience any murders. It’s only a big problem in certain urban areas. In 2014, the worst 2 percent of counties accounted for 52 percent of the murders. Five percent of counties accounted for 68 percent of the murders. Even within these counties, there are large regions without any murders.

Clearly, drug gangs have contributed a lot to the violent crime problems in America’s cities. Drug dealers use guns to protect their very valuable property, and they supply guns just like they sell drugs. Unfortunately, it is just as difficult to stop drug dealers from getting a hold of guns as it is to stop them from getting a hold of illegal drugs.

The popular press likes to compare crime rates in different places at the same point in time. But academics are aware of the limitations of this simple, cross-sectional comparison. Gun control advocates often compare the US and the UK, pointing out that the UK has stricter gun control and lower homicide rates than the US. Omitted is the fact that the UK’s homicide rate went up relative to the US after its gun control laws were enacted.

The UK’s homicide rate was still lower than the US’s, but it was despite the country’s counterproductive gun control laws, not because of them. The homicide rate was very low even before the UK had any gun control laws. To understand the effects of the laws, we have to see how homicide rates change before and after their implementation. Then, we can compare these changes in crime rates with the changes in places that didn’t reform their laws.

One thing gun control advocates such as Vox would never mention is that every single time that guns are banned — either all guns or all handguns — homicide/murder rates rise. This is a remarkable fact. One would think that just due to random chance, one or two countries would have a drop in homicides after banning guns.

Vox begins its discussion on mass public shootings using data collected by Jaclyn Schildkraut of the State University of New York-Oswego and H. Jaymi Elsass, a researcher at Texas State University. Unfortunately, in December 2015, when it was pointed out that their list was missing a lot of cases, Washington Post “Fact Checker” reporter Michelle Lee wrote Dr. Lott: “[Schilkraut] said they are still adding cases, and that it’s not a complete database.” However, Schilkraut and Elsass had already gone public with their findings about how the U.S. compared to other countries. They did so with full knowledge that they were missing many shootings in foreign countries. Not only did they miss cases, but Vox also doesn’t present the numbers for different countries on a per capita basis. It is startling that Vox puts other numbers in per capita terms, but not these numbers.

Here is a much more complete list of mass public shootings in the US. For cases around the world, see here.

Responding to Vox’s and the New York Times’ graphs.

1) Vox.com’s Claim #1 and New York Times Claim #1: “America has six times as many firearm homicides as Canada, and nearly 16 times as many as Germany”

Vox offers no explanation for why it compares only these 14 countries or why the New York Times looks at just 11. The New York Times graph is also mislabeled, as it clearly concerns firearm homicides, not murders. OECD, the organization of developed countries, has 34 member nations. But 192 countries worldwide provide homicide data.

Here are homicide rates across all countries for which data is available (click on figures to enlarge). Even disregarding the problems with homicide data being underreported in many countries, the US homicide rate is less than the median rate and half of the mean average for all countries (the raw data for this section are available here).

Now let’s look at the much smaller number of countries that report firearm homicide rates. The US rate looks much higher relative to other countries, but that is primarily because the countries with the highest homicide rates are the ones that don’t report their firearm homicide rates. Indeed, about 45% of the countries that report homicide numbers don’t report firearm homicide data.

Here is how homicide rates vary according to OECD numbers. The countries with the highest homicide rates don’t even report firearm homicides, and these same countries have very strict gun control regulations.

The US homicide rate is high, though the most important question here is how homicide rates vary with gun ownership. Vox doesn’t address this until point #6. Since it pertains to the homicide data that we have just discussed, we will address it here.

2) Claim #6: “It’s not just the US: Developed countries with more guns also have more gun deaths”

Here is the same figure for homicides generally, rather than just firearm homicides. Brazil, Mexico, and Russia are excluded. It uses the Small Arms Survey numbers despite the survey’s dramatic underestimation of gun possession rates in countries such as Israel and Switzerland, where guns possessed for decades by individuals are technically owned by the government and thus not counted. Whether or not someone possesses a gun is presumably what matters here. The Small Arms Survey also refuses to provide their list of sources for about 85% of the countries for which they present numbers. We will later show how these problems bias the results towards what Vox, Kristof, and the BBC want to show.

Excluding the US, it’s not true that there is an association between more guns and more gun deaths. In fact, higher gun ownership rates are associated with lower homicide rates.

Even with the US, there is a negative association once Brazil and Russia are included.

Some might object to including OECD-member Mexico as a developed country, but including it produces an even more negative relationship.

Similarly, increasing the reported number of firearms per 100 people for Israel and Switzerland (to reflect the possession of guns at home) also makes this relationship negative even when the US is included and Brazil, Mexico, and Russia are excluded.

When we look at all of the surveyed countries, the Small Arms Survey shows an association between more guns and fewer homicides.



The same is true for the much smaller set of countries that report firearm homicides.

3) Vox.com’s Claim #2: “America has 4.4 percent of the world’s population, but almost half of the civilian-owned guns around the world”

As mentioned earlier, countries such as Israel and Switzerland have a lot more civilian-possessed guns than civilian-owned guns. In these two countries, the household gun possession rate is certainly higher than in the US. But there is a major accuracy problem in using surveys or registration lists to count gun ownership. There is strong evidence that most guns are never registered.

When Canada tried in the late 1990s to register its estimated 15 million to 20 million long guns, about 7 million were actually registered. In the 1970s, Germany registered 3.2 million of the country’s estimated 17 million guns. In the 1980s, England registered only about 50,000 of the estimated 300,000 pump-action and semiautomatic shotguns in the country.

Even in the US, there is evidence that surveys of gun ownership rates are not very accurate. In many countries where gun ownership is illegal, surveys continually show zero gun ownership even when that is clearly not the case.

The bottom line is that the number of guns in the rest of the world is underestimated relative to the number in the United States. So 42% is likely a huge overestimate of the US’s true share of guns worldwide.

4) Vox.com’s Claims #3 and #4: “There have been more than 1,600 mass shootings since Sandy Hook” and “On average, there is around one mass shooting for each day in America”

As Vox notes: “The tracker uses a fairly broad definition of ‘mass shooting’: It includes not just shootings in which four or more people were murdered, but shootings in which four or more people were shot at all (excluding the shooter).” This definition comes from a research group called the Gun Violence Archive. The FBI has used two different definitions: mass shootings and mass public shootings. Besides including the much more numerous cases where no one was killed, Vox’s definition of a mass public shooting differs dramatically from the traditional FBI definition in other ways. The FBI excludes gang fights in order to focus on those cases where the point of the attack was to kill people. The idea is to focus on the types of mass public shootings that we see at schools, malls, and other public places. This isn’t to say that gang fights over drug turf are less important, but the causes and solutions for them are dramatically different than for the mass public shootings that we hear about on the news.

Here is an earlier CPRC post on miscounting of mass public shootings.

5) Vox.com’s Claim #5: “States with more guns have more gun deaths” and New York Times’ Claim #3: “Fewer Guns = Fewer Deaths”

These figures use purely cross-sectional data, looking at all of these states at just one point in time. But to do a proper analysis, one has to see how crime rates vary over time across all of the states. Did the states that experienced the biggest increases in gun ownership also have the biggest increases or decreases in crime rates? Here is part of a discussion from Dr. John Lott’s “More Guns, Less Crime” (University of Chicago Press, 2010). We haven’t taken the time to graph this out yet, but if we had, the graph would look very similar to the one shown below in response to Claim #5.

Other points raised by the New York Times on this issue:

“One study by the Violence Policy Center found that in 2012 there were 259 justifiable homicides by a private citizen using a firearm.”

This number is a huge underestimate, but the Times makes no attempt to address common objections to the claim. Few jurisdictions report justifiable homicide numbers to the FBI, and whether a case is justifiable or not is often not determined until after a trial. Even those police departments that do report justifiable self-defense cases don’t bother to go back and correct earlier numbers. Furthermore, the FBI definition is very limited, only counting a homicide as justifiable if it occurred in conjunction with another offense.

“32% of households have guns”

In another example of bias, the Times claims that only 32% of American households own guns. That number comes from the General Social Survey (GSS), but it is an outlier. A March 2018 poll by NBC News and the Wall Street Journal estimates that 47% of households own guns, with another 3% declining to answer. A Monmouth University Poll on March 2-5, 2018 asked: “Do you or anyone in your household own a gun, rifle, or pistol?” With 46% saying “yes” and another 7% unsure or refusing to answer, it is quite plausible that half of all households own guns.

6) Vox.com’s Claim #7: “America is an outlier when it comes to gun deaths, but not overall crime”

Vox uses the graph to support its claim that: “The US appears to have more lethal violence — and that’s driven in large part by the prevalence of guns.”

The United States has a relatively low violent crime rate compared to other developed countries. The graph below includes the average violent crime rate across these countries (a separate breakdown by sexual assault, robbery, and assault is available here). Compared to those countries, the United States does have a relatively high homicide rate. But Vox doesn’t discuss the most obvious explanation: that the US has a bad drug gang problem. Over half of US murders occur in just 2% of the counties, and the murders are heavily concentrated in small areas within those counties.

7) Vox.com’s Claim #8: “States with tighter gun control laws have fewer gun-related deaths”

This graph is from Vox.com. Kristof illustrates this differently, but the point is the same: states with the most gun control laws or the lowest gun ownership rates have the lowest gun death rates (firearm homicides + firearm suicides).

As with our earlier discussions, purely cross-sectional comparisons can be very misleading. A detailed discussion is available here. The graph below is the result when you account for pre-existing differences across states and national changes in crime rates. It shows a positive association between death rates and the number of gun control laws.

8) BBC claim #5: Attacks in US become deadlier

To prove this claim the BBC provides this diagram for the US over the period from 1991 to 2018.

What if we were to put together the same type of chart for the US and Western Europe. The first graph looks at the 10 worst cases in terms of murders, though the Near Stirling, Scotland and Parkland, Florida attacks are tied for tenth. The second graph includes all the cases where at least 13 people were killed. Using the type of graph that the BBC uses it would appear as if the BBC could have a graph that says: “Attacks in Western Europe become deadlier at a faster rate than in the United States.”

But the US is far from having the deadliest attacks and the number of attacks and how deadly they are is increasing much faster in the rest of the world (research available here).

9) New York Times’ Claim #6: More on their claim that “Tightening Gun Laws Lowered Firearm Homicide Rates”

It makes little sense to examine one state at a time when ten states have laws which require gun ownership licenses under at least some circumstances (Hawaii, Illinois, Iowa, Missouri, Massachusetts, Michigan, Nebraska, New Jersey, New York, North Carolina, and the District of Columbia). Others have expanded background checks. Instead, the New York Times looks only at Connecticut and Missouri, selectively picking out years and what crime rates to look at, then using different tests and different controls.

For example, in the three of the four years following implementation of Connecticut’s gun licensing law in 1995, the state firearm homicide rate rose relative to the other Northeastern States. But there is no theory offered for why Connecticut’s firearm homicide rate would first rise relative to other Northeastern states, then fall relatively for six years, and finally rise relatively for four of the next five years. Yet, the study of Connecticut compares the state’s firearm homicide rate right before the law was implemented with its lowest subsequent point relative to other states.

A much more detailed discussion of Connecticut is available here, along with a detailed discussion of how other types of crime rates changed.

Unlike the study for Connecticut, which compares two points in time, the study for Missouri looks at before-and-after average crime rates. That can be totally misleading. While it is true that after Missouri’s law was changed, the state’s murder rates rose 17 percent relative to the rest of the US, it had actually increased by 32 percent during the five years prior to the change. So the Missouri murder rate increased relative to the rest of the US at a slower rate after the change in the law than it did prior to it. The more pertinent question here is why the rate of increase slowed down.

The period 1999 to 2012 is used for the graph below because that is the period of time examined in Webster’s study. The data for Missouri is available here and for the US here. The Missouri numbers are not included in calculating the US rates.

A much more detailed discussion on Missouri is available here.

10) New York Times’ Claim #2: “We Have a Model for Regulating Guns: Automobiles”

Kristof believes that automobile regulation serves as a model for gun regulations. He argues that automobile safety regulations saved lives and that gun control can do the same.

Kristof’s graph makes it appear as if the drop in motor vehicle deaths really only started to occur after the first seatbelts were provided in cars. As indicated by the figures from Dr. Lott’s book, The War on Guns: 1) using the same data source as Kristof, it is easy to see that cars were getting safer from the time the very first data was released in 1921 and 2) when you look over the entire period, the rate at which car safety improved actually slowed down after the federal government started regulating car safety. The first seatbelts were introduced in 1950 by car companies that were figuring out on their own how to make cars safer. It is unfortunate that the New York Times’ graph doesn’t show the even faster drop in vehicle deaths per-mile-traveled that occurred before 1946.

Unfortunately, government delays in approving safety products haven’t just affected automobile safety. These delays have probably also increased deaths and injuries from skin cancer.

11) Vox.com’s Claim #9: “Still, gun homicides (like all homicides) have declined over the past couple decades”

This is correct.

12) Vox.com’s Claim #10: “Most gun deaths are suicides”

Most firearm deaths are indeed suicides. But Vox lumps together murders and justifiable homicides into the category of “firearm homicides.” Obviously, justifiable homicides should be counted differently from murders. Excluding justifiable homicides implies that firearm suicides make up about 70% of firearm deaths. (For murder data, see here, and for suicides and accidental deaths see here.)

13) Vox.com’s Claim #11: “The states with the most guns report the most suicides”

Once again, Vox’s use of purely cross-sectional comparisons can be very misleading. Vox tries to show that states with high gun ownership have high suicide rates, but they ought to consult a notable economics paper on suicide by Cutler, Glaeser, and Norberg. Cutler, et al. found that rural areas have both a large male-female population imbalance and also more gun ownership. They found the real cause to be the high number of partnerless, older men.

Instead of relying on surveys of gun ownership, we can use the number of concealed handgun permits as a proxy for ownership. And when we follow states over time, we see no association between suicides and the number of concealed handgun permits.

There is a strong tendency for states with more gun control laws to have lower gun ownership rates. Yet, doing this research properly, we find that having more gun laws is associated with more total suicides and is unrelated to firearm suicides.

14) Vox.com’s Claim #12: “Guns allow people to kill themselves much more easily”

Vox argues: “Perhaps the reason access to guns so strongly contributes to suicides is that guns are much deadlier than alternatives like cutting and poison.”

But Vox gives a very misleading impression of the effectiveness of different suicide methods. A 1995 study looked at 4,117 cases of completed suicide in Los Angeles County during the period 1988-1991, and found that the success rate for being hit by a train is virtually the same as for a gunshot to the head or a shotgun to the chest. The study also estimated that the amount of pain and discomfort from being hit by a train was about half that of the other two methods. (Click on the pie charts to enlarge.)

The second problem with these numbers is that not everyone wants to successfully commit suicide, so people can affect the success rate of the method. They may take a few extra pills, but not enough to actually kill themselves.

15) Vox.com’s Claim #13: “Policies that limit access to guns have decreased suicides”

Vox cites two studies which claim that gun control laws can reduce suicides. One study is on Australia’s 1996/97 gun buyback and another is on Israel. But cherry-picking two studies isn’t very useful. Vox just ignores research that doesn’t support their conclusions.

This is part of a longer piece that Dr. John Lott wrote up on Australia’s buyback program (an entire chapter in “The War on Guns” covers this topic; the data are available here):

But looking at simple before-and-after averages of gun deaths in Australia regarding the gun buyback is extremely misleading. Firearm homicides and suicides were falling from the mid-1980s onwards, so you could pick out any subsequent year and the average firearm homicide and suicide rates after that year would be down compared to the average before it. The question is whether the rate of decline changed after the gun buyback law went into effect. But the decline in firearm homicides and suicides actually slowed down after the buyback. Australia’s buyback resulted in almost 1 million guns being handed in and destroyed, but after that private gun ownership once again steadily increased and now exceeds what it was before the buyback. In fact, since 1997 gun ownership in Australia grew over three times faster than the population (from 2.5 million to 5.8 million guns). Gun control advocates should have predicted a sudden drop in firearm homicides and suicides after the buyback, and then an increase as the gun ownership rate increased again. But that clearly didn’t happen. . . .

The Israeli study is poorly done. Starting in 2006, Israeli soldiers were no longer allowed to take their guns home with them on weekends. There was a drop in suicides in 2007-2008 compared to 2003-2005, but a better study would have compared soldiers who could have taken their guns home with those who couldn’t. The study uses only simple before-and-after averages, similar to the Australian comparisons. No month-to-month changes are provided, so it isn’t clear exactly when the drop in suicides started.

A similar issue has already been tested in the proper way. Permitted concealed carry is analogous to soldiers being able to take their guns home with them. And the research finds no increase suicides when people are allowed to carry their handguns with them.

Vox selectively picks research and ignores the reality that some gun control regulations, such as gunlock laws, actually cause an increase in total deaths. In the case of locks, guns are made less accessible for self-defense.

16) Vox.com’s Claim #14: “In states with more guns, more police officers are also killed on duty”

Vox cites a study in the American Journal of Public Health which claims that states with more guns also have more cops die in the line of duty. But this study is particularly flawed. There are normal controls that researchers use to account for average differences across places and across years, and this study only accounts for the average differences across places. If both factors were accounted for, as must always done in this kind of research, the authors would have gotten the opposite results from what they claim.

17) Vox.com’s Claim #15: “Support for gun ownership has sharply increased since the early 2000s”

There are a number of surveys that show support for gun control peaked around 1998 and 1999. The Pew survey that Vox cites is just one such survey. Why they cut off the survey data in 2000 and don’t go back further is a puzzle that only Vox can answer. The Gallup and CNN surveys are available here.

18) Vox.com’s Claim #16: “High-profile shootings don’t appear to lead to more support for gun control in the long term”

The key here is “long term.” It is the reason why gun control groups try to push through gun control in the immediate aftermath of shootings.

19) Vox.com’s Claim #17: “Specific gun control policies are fairly popular” and New York Times’ Claim #9: “A Way Forward: On Some Issues, Majorities Agree”

Given the supposed 90% support that the media tells us exists for expanded background checks, one would think that Michael Bloomberg’s well-funded ballot initiatives in 2016 in Nevada and Maine would have been slam dunks. Yet, Bloomberg lost in Maine by 4 percent and won in Nevada by just 0.8 percent. The initiative only eked out the win in Nevada because of the false promise that it wouldn’t cost the state anything and because of the $20 million spent in support of it (amounting to an incredible $35.30 per vote). Bloomberg outspent his opponents by a factor of three – in Maine, the $8 million he spent outdid the other side by a factor of six.

20) New York Times’ Claim #7: “There Is a Shocking Lack of Research on Guns”

As our discussion of Claim #6 points out, the research by public health researchers is very poorly done. Indeed, it seems to be driven by political goals. For the government to fund this research is no better than giving money directly to Bloomberg’s gun control organizations.

21) New York Times’ Claim #8: “The Right Type of Training Could Go a Long Way”

The issue is ultimately how people perform once they get training. Dr. John Lott finds that concealed handgun permit holders rarely get into any type of trouble with their concealed handguns, with their permits being revoked for any reason at rates of thousandths or tens of thousandths of one percent. Permit holders are convicted of firearms violations at a lower rate than police are, and the revocation rate has remained stable over many years (“More Guns, Less Crime” [University of Chicago Press, 2010]).

22) New York Times’ Claim #4: “Mass Shootings Are Not the Main Cause of Loss of Life”

While Kristof mentions the Las Vegas attack as an example of a mass shooting, the 456 mass shootings that are counted in the graphic have little to do with the mass public shootings that capture media attention. As with Vox.com, these numbers are based on the Gun Violence Archive.

The self-defense numbers are based on the justifiable homicides by a private citizen using a firearm, and, as we discussed above, these numbers are fatally flawed.

23) New York Times’ Claim #5: “America Is Moving in the Wrong Direction”

We believe that states have actually been moving in the right direction on open and concealed carry.

Facebook Twitter LinkedIn

CPRC original research, Fact Checking Bias, gun control advocates, Gun Control Lies

By johnrlott

