Cargo Cult Science

We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work. And it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.

Cargo Cult Science by Richard Feynman 1974

I encourage you to take the time and listen to what Feynman has to say and if you’re so inclined also read his piece on Cargo Cult Science.

Before I continue, I want make it crystal clear that:

I do not dismiss social sciences outright.

dismiss social sciences outright. I regard all research & findings of respected, peer-reviewed journals equally (such as Thomson Reuters’ Master Journal List)

research & findings of respected, peer-reviewed journals equally (such as Thomson Reuters’ Master Journal List) I will even acknowledge the importance/findings of peer-reviewed journals that might not be on the MJL.

With that being said though; I do consider social sciences to be by definition less precise than for example the natural sciences. The reason I say by definition, is because humans are almost incomprehensibly complex compared to for example the experiments done in a particle accelerator. Which in effect means that it will never suffice for me to just read abstract of a paper, I will always demand the entire paper in order for me to fully acknowledge a finding.

You might be asking yourself by now if I have completely lost the plot and how this all is ever going to relate back to GamerGate. Well I already alluded to the what I want to expand on now with the first picture in this post. And I now want to elaborate further on it with this (marvellous) drawing here

We’re on a boat B)

What I’m trying to say with this drawing, is that I believe the reason we are dealing with all of this, on a global scale, as well as within GamerGate, is because people haven’t been very good at critically examining the results coming from fields such as social psychology. And so we are now faced with the situation that not only has this research crept into the education system, it is also in the heads of the journalists, developers, adbusters and people in general.

If you want more than just my word (and I hope that you demand more when someone makes such claims), here are some examples and facts to back up my claims:

Social Psychology suffers particularly from the so-called Replication Crisis as detailed in its Wikipedia-article. Since it is the basis for all of the claims that Anita Sarkeesian makes (assuming she even bothers to quote any science) in her videos, I want to include it here in full:

Social psychology has recently found itself at the center of a “replication crisis” due to some research findings proving difficult to replicate. Replication failures are not unique to social psychology and are found in all fields of science. However, several factors have combined to put social psychology at the center of the current controversy. Firstly, questionable researcher practices (QRP) have been identified as common in the field. Such practices, while not intentionally fraudulent, involve converting undesired statistical outcomes into desired outcomes via the manipulation of statistical analyses, sample size or data management, typically to convert non-significant findings into significant ones.[33] Some studies have suggested that at least mild versions of QRP are highly prevalent.[34] One of the critics of Daryl Bem in the feeling the future controversy has suggested that the evidence for precognition in this study could (at least in part) be attributed to QRP. Secondly, social psychology has found itself at the center of several recent scandals involving outright fraudulent research. Most notably the admitted data fabrication by Diederik Stapel[35] as well as allegations against others. However, most scholars acknowledge that fraud is, perhaps, the lesser contribution to replication crises. Third, several effects in social psychology have been found to be difficult to replicate even before the current replication crisis. For example the scientific journal Judgment and Decision Making has published several studies over the years that fail to provide support for the unconscious thought theory. Replications appear particularly difficult when research trials are pre-registered and conducted by research groups not highly invested in the theory under questioning. These three elements together have resulted in renewed attention for replication supported by Kahneman. Scrutiny of many effects have shown that several core beliefs are hard to replicate. A recent special edition of the journal Social Psychology focused on replication studies and a number of previously held beliefs were found to be difficult to replicate.[36] A 2012 special edition of the journal Perspectives on Psychological Science also focused on issues ranging from publication bias to null-aversion that contribute to the replication crises in psychology[37] It is important to note that this replication crisis does not mean that social psychology is unscientific. Rather this process is a healthy if sometimes acrimonious part of the scientific process in which old ideas or those that cannot withstand careful scrutiny are pruned.[38] The consequence is that some areas of social psychology once considered solid, such as social priming, have come under increased scrutiny due to failed replications[39]

I hope you haven’t forgotten that I stated earlier that this does not mean that I dismiss the entire field outright. I don’t. I consider all scientific endeavors important. But from everything I have seen coming from (mostly) social sciences, I have learned to take it with more than a pinch of salt.