A typical experiment may compare the behavior of two groups of college students, one group that plays violent video games and one group that plays only nonviolent video games. After a period of game-playing, the students are told that someone who does not like hot sauce has agreed to eat it. Each participant gets to choose the quantity of hot sauce. In a 2014 study at the University of Sussex, those who played violent video games doled out more hot sauce than those who did not. Are they more aggressive? Are they more violent? Would an extreme version of this behavior consist of buying a rifle and shooting up their high school?

The link between aggression and violence—especially the kind recently on display in Florida’s Parkland shooting—is tenuous at best. While aggression is widespread, and often admired (football anyone?), violence is much less ubiquitous, especially the kind of violence that goes beyond a fist fight, and into the realm of mass shootings.

But even if we agree that voluntarily subjecting someone to spicier hot sauce indicates increased aggression, there is plenty of confusion about what constitutes a violent video game. According to Craig Anderson, a professor at Iowa State University, who champions a link between violence and violent gaming,

“Violent media are those that depict intentional attempts by individuals to inflict harm on others. An ‘individual’ can be a nonhuman cartoon character, a real person, or anything in between.”

As a result, most video games are violent—from the obvious, such as Call of Duty: WWII or Destiny 2, to the less obvious. Mario Party 9 is considered violent, because the characters purposely hop on other characters’ heads to lay them flat. Games used in research on the violent-game-aggressive-behavior link include Ty2, featuring a cartoon Tasmanian tiger in a customizable world (who nonetheless throws boomerangs against his enemies).

And therein lies a big problem. The research is plagued by explanations that don’t isolate “violence” as a factor by itself. I may respond more aggressively after playing Mario Party 9 than after playing a cooperative game, but the response might be about excitement, adrenalin, or competition, rather than violence. In a small study out of Brock University in Ontario, students who played more competitive games doled out more and hotter sauce, but equally competitive games with different levels of violence did not result in aggression measured in terms of hot sauce.

When it comes to controlled environments, “violent video games” and “nonviolent video games” have more than just “violence” as differences. Mismatched games make for poor comparisons; even if people exposed to the violent games exhibit more aggression, the result is up for interpretation.

The outcomes also have questionable clinical significance. We don’t even know how meaningful the differences in the heat of hot sauces are. Does a difference of 100 Scoville Heat Units (SHU) matter? What about 1000? How would we compare those different “aggression levels” across different studies, some of which use hot sauce (perhaps without recording the heat) as a surrogate for aggression, and some of which use loudness of a noise participants choose to subject someone to after playing video games?

Most studies aiming to assess video game impact are small, so researchers have turned to meta-analyses to combine studies with hopes of more conclusive results. But a meta-analysis relies on studies being comparable; how does one match up an aggression surrogate like the quantity of hot sauce a participant puts in a cup, and another metric of aggression like how loud a participant makes a punishing sound? Incomparable results and disagreement about the interpretation of these results plague contrasting meta-analyses from giants in the field of video game violence and its impact (such as two meta-analyses on violent video games and linked aggression, one by CJ Ferguson at Stetson University, and one by Anderson).

Observational studies have the advantage that they can find people who have exhibited actual violent behaviors. In one study, kids ages 9-18 who played violent video games in the last year were more likely to report bringing a weapon to school in the last month than kids who had no exposure; yet, these same kids were more likely to be bullied and feel threatened, possibly bringing in weapons for protection rather than from an intent to harm. A study looking at student mass-killer assailants found only one in eight students had an interest in violent video games, which is less than the interest exhibited by people who have not committed the same acts.

Different problems plague observational studies: kids who are attracted to violent video games may already be inclined toward violence. Which came first?

The question of violence in media is not new, even if the medium of video games is comparatively new. We used to worry about violence on television and in movies. Consider the massive scale with which we consume these media, and it’s hard to justify the conclusion that the game industry is to blame for mass shootings. If violent video games led directly to such violent actions—and more directly than, say, access to guns—we would see even more such shootings. While video game consumption has gone up, mass shootings (as a proportion of the population) has stayed constant. Unfortunately, the number of deaths for each mass shooting has gone up.

Updated, March 12, 2018