Last year, Peter Boghossian and James Lindsay successfully published a nonsense paper that was titled “The Conceptual Penis As A Social Construct” in the journal Cogent Social Sciences. They argued this provided evidence that gender studies as a discipline required “serious housecleaning.” However, I argued that they overstated their conclusion in this post because getting one bad paper published in one bad journal (that requires payment to publish) doesn’t provide much evidence that an entire field is in trouble. Alan Sokal of the famous Sokal Hoax also thought the “Conceptual Penis Hoax” was too limited to justify the author’s claims.

To Boghossian and Linday’s credit, they listened to criticism and attempted a far more comprehensive approach this time. They also added Helen Pluckrose to the new attempt as well. Pluckrose, Lindsay, and Boghossian wrote 20 papers and submitted them to much higher quality journals in the fields of gender studies, cultural studies, and critical theory. They lumped these fields together under the name”grievance studies” because “their common goal of problematizing aspects of culture in minute detail in order to attempt diagnoses of power imbalances and oppression rooted in identity.”

Their goal was to expose poor scholarship and ideological bias in these fields. You can read their extensive overview of their methods here. I’m going to share some excerpts of theirs followed by my thoughts as a sociologist. First, they describe their approach and objective:

Our approach is best understood as a kind of reflexive ethnography—that is, we conducted a study of a peculiar academic culture by immersing ourselves within it, reflecting its output and modifying our understanding until we became “outsiders within” it. Our objective was to learn about this culture and establish that we had become fluent in its language and customs by publishing peer-reviewed papers in its top journals, which usually only experts in the field are capable of doing. Because we came to conceptualize this project as a kind of reflexive ethnographic study in which we sought to understand the field and how it works by participating in it, obtaining peer reviewers’ comments about what we were doing right and what needed to change to make absurd theses acceptable was central to the project. Indeed, the reviewers’ comments are in many ways more revealing about the state of these fields than the acceptances themselves.

Okay, that’s straightforward enough. They read about these fields a lot, thought they were absurd, and tried to see if they could reproduce absurd findings in decent academic journals. Next they go into the paper writing methodology:

Our paper-writing methodology always followed a specific pattern: it started with an idea that spoke to our epistemological or ethical concerns with the field and then sought to bend the existing scholarship to support it. The goal was always to use what the existing literature offered to get some little bit of lunacy or depravity to be acceptable at the highest levels of intellectual respectability within the field. Therefore, each paper began with something absurd or deeply unethical (or both) that we wanted to forward or conclude. We then made the existing peer-reviewed literature do our bidding in the attempt to get published in the academic canon. This is the primary point of the project: What we just described is not knowledge production; it’s sophistry. That is, it’s a forgery of knowledge that should not be mistaken for the real thing. The biggest difference between us and the scholarship we are studying by emulation is that we know we made things up.

So they did justify their absurd claims with some real research in the field to build legitimacy. They go into more detail about their process here:

This process is the one, single thread that ties all twenty of our papers together, even though we used a variety of methods to come up with the various ideas fed into their system to see how the editors and peer reviewers would respond. Sometimes we just thought a nutty or inhumane idea up and ran with it. What if we write a paper saying we should train men like we do dogs—to prevent rape culture? Hence came the “Dog Park” paper. What if we write a paper claiming that when a guy privately masturbates while thinking about a woman (without her consent—in fact, without her ever finding out about it) that he’s committing sexual violence against her? That gave us the “Masturbation” paper. What if we argue that the reason superintelligent AI is potentially dangerous is because it is being programmed to be masculinist and imperialist using Mary Shelley’s Frankenstein and Lacanian psychoanalysis? That’s our “Feminist AI” paper. What if we argued that “a fat body is a legitimately built body” as a foundation for introducing a category for fat bodybuilding into the sport of professional bodybuilding? You can read how that went in Fat Studies. At other times, we scoured the existing grievance studies literature to see where it was already going awry and then tried to magnify those problems. Feminist glaciology? Okay, we’ll copy it and write a feminist astronomy paper that argues feminist and queer astrology should be considered part of the science of astronomy, which we’ll brand as intrinsically sexist. Reviewers were very enthusiastic about that idea. Using a method like thematic analysis to spin favored interpretations of data? Fine, we wrote a paper about trans people in the workplace that does just that. Men use “male preserves” to enact dying “macho” masculinities discourses in a way society at large won’t accept? No problem. We published a paper best summarized as, “A gender scholar goes to Hooters to try to figure out why it exists.” “Defamiliarizing,” common experiences, pretending to be mystified by them and then looking for social constructions to explain them? Sure, our “Dildos” paper did that to answer the questions, “Why don’t straight men tend to masturbate via anal penetration, and what might happen if they did?” Hint: according to our paper in Sexuality and Culture, a leading sexualities journal, they will be less transphobic and more feminist as a result.

Here are a few of their paper topics that had some dubious ethics:

Many papers advocated highly dubious ethics including training men like dogs (“Dog Park”), punishing white male college students for historical slavery by asking them to sit in silence in the floor in chains during class and to be expected to learn from the discomfort (“Progressive Stack”), celebrating morbid obesity as a healthy life-choice (“Fat Bodybuilding”), treating privately conducted masturbation as a form of sexual violence against women (“Masturbation”), and programming superintelligent AI with irrational and ideological nonsense before letting it rule the world (“Feminist AI”). There was also considerable silliness including claiming to have tactfully inspected the genitals of slightly fewer than 10,000 dogs whilst interrogating owners as to their sexuality (“Dog Park”), becoming seemingly mystified about why heterosexual men are attracted to women (“Hooters”), insisting there is something to be learned about feminism by having four guys watch thousands of hours of hardcore pornography over the course of a year while repeatedly taking the Gender and Science Implicit Associations Test (“Porn”), expressing confusion over why people are more concerned about the genitalia others have when considering having sex with them (“CisNorm”), and recommending men anally self-penetrate in order to become less transphobic, more feminist, and more concerned about the horrors of rape culture (“Dildos”). None of this, except that Helen Wilson recorded one “dog rape per hour” at urban dog parks in Portland, Oregon, raised so much as a single reviewer eyebrow, so far as their reports show.

Wow. Yes, pretty outrageous topics. And again, they just made these up before reading literature that could support the claim. So what was the verdict of these 20 papers?

Seven were published in peer-reviewed journals. Six were rejected. The other 7 were still in limbo when they got caught. You can check out which papers were published where in their full report. And they even uploaded the full papers and reviewer comments online here. Not only were these nonsense articles published, they received a good amount of praise from the reviewers.

My first thought was to check out the journals that published such absurd manuscripts. They are indeed legitimate journals in fields of gender studies and feminist theory. Most of them have been around for decades and have impact factors that hover around 1 or 2. Impact factor is a measure of how often articles in the journal are being cited by other articles. So for smaller niche areas like these, a 1 or 2 makes sense. Interestingly, the three sociology focused journals all rejected the papers sent to them.

So as a critic of the original “Conceptual Penis” hoax, I have to say this second attempt was much more comprehensive and supportive for their main point. Yes, these subfields appear to publish some pretty wacky papers that lack substantial academic rigor.

As a sociologist, I’m fine with exposing the poor scholarship. As far as I can tell, this hoax illustrates a larger known issue with some areas of social science and the humanities: a lack of concise, logical, and cumulative theory.

I only skimmed through the papers, but the format is similar to some papers I’ve previously read in these types of subfields. The writing is long-winded and difficult to comprehend. Instead of theory that is comprised of concise, logical propositions, we see a confusing mess of ‘stream of consciousness’ writing with citations peppered in. If a journal publishes articles with obscure wording and confusing theoretical support, then it will be vulnerable to bad scholarship.

Pluckrose, Lindsay, and Boghossian suggest that universities monitor these subfields and make sure they are producing rigorous scholarship. I’m not exactly sure how any external impact would happen though. I believe the change has to come from within.

Sociologists have already been advocating for more rigorous theory building in their field. And there is plenty of rigorous social science out there. Something I wish the “grievance studies” project would have included are good examples of scholarship in identity and gender studies. This could provide a useful comparison.

Expectation States Theory is a great example of a rigorous theory that allows us to test potential group advantages in society. In this overview, you can read how concepts like race and gender are conceptualized and studied in a direct and testable way. As for identity, Affect Control Theory is another rigorous theory that allows us to explain behavior in the context of social interactions.

Both of these examples are within social psychology and often deal with experiments. However, you don’t have to do experimental work to be rigorous. You can write a theory paper that builds off other theory in a meaningful way. In order to do that, we should strive for transparency and clarity instead of vague and obscure writing. By doing so, it makes it much easier to identify bad scholarship because it is more obvious when it doesn’t advance the field.

So to summarize this already lengthy post, yes, I think the “grievance studies” project illustrates some important points. Academic peer-review has issues, but it’s still the best system we have right now. The problems raised here deal more with what type of work is deemed acceptable among the academic peers. The scholars of these fields should be open to improving the level of scholarship in their fields. There already are many academics in the social sciences who want to improve our theory building and methodological practices. I hope more and more scholars embrace the change and understand that greater academic rigor will only help our legitimacy in the long term.

PS: I now have a Patreon if you’d like to support my writing and podcasting.