In general, we are concerned that the study of “sexist” video games may be following the path of research about video games and violence. This includes using emotionally evocative labels for games, inflated rhetoric regarding the strength and consistency of effects, exaggerated press releases, and weak research designs (Ferguson 2013b; Hall et al. 2011; Markey et al. 2015). The violent video game field also appears to be touched by psychology’s replication crisis, with direct replications of older studies ultimately failing to confirm previous results (e.g., Przybylski et al. 2014; Tear and Nielson 2013, 2014) and suggestions of publication bias in the existing literature (Hilgard et al. 2017). We worry that a similar state of affairs is emerging for “sexist” video game research, and it may be a facet of the larger moral panic facing video games and games research (Bowman 2016; Ferguson 2013b). We also understand that there is occasionally good theater (and even fun) in academic debates and consider the response by Gabbiadini and colleges in such a context. Nonetheless it is important to move beyond theatrics so we conclude with some constructive comments for how the field and consumers of research about media effects on youth outcomes may move forward.

Preregistration

As we have indicated both in this reply and in our original article, preregistration has numerous benefits. It provides confidence that complex moderator/mediator analyses were planned in advanced and are not the result of HARKing or Garden of Forking Paths issues. The current back and forth between our respective groups would be much different if there was verifiable proof of the theoretical model initially guiding the original study as well as a detailed analytic plan.

We are also concerned that theories in media effects are often slippery and shifting when it comes to direct effects vs. moderated mediation and what variables count as mediators as opposed to ultimate dependent variables. We worry there is a lot of analytic flexibility with existing studies (e.g., Elson et al. 2014) and we suspect it is fairly easy to find at least some effect that is p < .05 (e.g., Simmons et al. 2011). Collectively, these conditions make many claims about media effects difficult to falsify. Preregistration along with a commitment to publishing null effects would help produce a literature with unbiased effect size estimates. That kind of literature may actually prove useful to parents and those concerned about youth development. Our fear is that the current literature is biased so that it is not especially well suited for drawing real world implications.

21-Word Solution

Simmons et al. (2012) suggest a simple disclosure statement that can be used to make sure all variables are reported in a published article. They call this a 21-word solution to some of the problems of analytic flexibility. The statement reads: “We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study” and can be included (when true) in the Method section of all articles. The inclusion of this statement in research would further help address concerns with analytic flexibility such as the specific concerns about the omitted reporting in Gabbiadini et al. (2016) we have identified. Consumers of studies can look to this as an additional signal of the quality of the work. The 21-word solution could not have been applied to the Gabbiadini et al. (2016) report as it was written.

Separating Advocacy from Science

We support advocacy pushing for better representations of female characters in video games, and salute some recent positive moves in this direction (e.g., the Tomb Raider reboot, Horizon Zero Dawn; Alice: Madness Returns, Portal; Going Home and Beyond Good and Evil.) We also believe that sexist attitudes and practices are deplorable. However, advocacy and science are distinct with different objectives and different evidentiary requirements. Advocacy is about changing practices and attitudes whereas sciences is ultimately about figuring out reality. Advocates often emphasize information that supports a particular goal while they may deemphasize or even omit information that does not support a particular position. Advocacy can be fueled by explicitly moral agendas. Science searches for truth however convenient or inconvenient for any particular agenda or perspective. In many cases, combining advocacy with science may prove detrimental to both efforts.

Advocacy is important for drawing attention to sexist representations in games and motivating designers to change the depictions of women in games. Likewise, pointing to disparities in gender representation among game designers, or the harassment faced by female gamers are worthwhile efforts. To the extent that advocates rest their arguments on the existence of causal media effects, they risk making claims based on shaky grounds. Concerns that evidence cited in these arguments are “cherry-picked” or discredited by other research could inadvertently harm well-intentioned advocacy efforts to the extent that they lose credibility.

We argue that science remains most effective when it remains neutral insofar as advocacy efforts are concerned. We understand that many scholars may wish to put their data to use in support of various efforts to better the human condition. However, we struggle to think of multiple examples where mixing advocacy with science does not damage the objectivity of the latter. This has been a verified problem for some video game violence research where some scholars associated closely with or received research funding from anti-media advocacy groups (Ferguson 2013b). These mistakes should not be repeated with sexist media research.