The world as a whole has become increasingly reliant on science to provide its technology and inform its policy. But rampant conspiracy theories, fake news, and pseudoscience like homeopathy show that the world could use a bit more of the organized skepticism that provides the foundation of science. For that reason, it has often been suggested that an expanded science education program would help cut down on the acceptance of nonsense.

But a study done with undergrads at North Carolina State University suggests that a class on scientific research methods doesn't do much good. Instead, a class dedicated to critical analysis of nonsense in archeology was far more effective at getting students to reject a variety of pseudoscience and conspiracy theories. And it worked even better when the students got their own debunking project.

The study, done by Anne Collins McLaughlin and Alicia McGill, lumps together things like belief in astrology, conspiracy theories, and ancient aliens, calling them "epistemically unwarranted." Surveys show they're widely popular; nearly half the US population thinks astrology is either somewhat or very scientific, and the number has gone up over time.

You might think that education, especially in the sciences, could help reverse this trend, but McLaughlin and McGill have some depressing news for you. Rejection of epistemically unwarranted ideas doesn't correlate with scientific knowledge, and college students tend to have as much trouble coming to grips with reality as anyone else.

Rather than hope that a general exposure to academia would produce critical thinking skills, the authors took matters into their own hands, designing a course that used history and archeology to critically analyze a variety of epistemically unwarranted ideas that have kicked around for a while: the existence of Sasquatch, the Bermuda Triangle, mummies' curses, the lost continent of Atlantis, ancient aliens, and more. All of these were handled critically, with a focus on evidence, credibility of sources, and so on.

The course was so popular that McGill ended up teaching both a regular and honors version of it, with the honors group performing an independent analysis of a local legend, the Brown Mountain Lights. For a control, the authors used a class that should teach critical analysis by implication: research methods in psychology. Students in each of these classes were initially given a survey that provided a picture of their scientific knowledge, then were asked about a variety of pseudosciences and conspiracies.

(In addition to the topics covered in class, the questions touched on a variety of unwarranted beliefs: the Earth being 6,000 years old, the number 13 being unlucky, astrology, alternative medicine, crop circles, the Illuminati, and the Moon landing being a hoax.)

At the start of the semester, there was no difference in the level of epistemically unwarranted belief in the three classes. There was also no correlation between general scientific knowledge and these unwarranted beliefs.

When the semester ended, however, matters were rather different. Those who went through the research methods class showed no change in their level of unwarranted belief. Although this undoubtedly included some techniques that could help identify these beliefs, there's no indication that the students applied them without explicit instructions to do so.

By contrast, the students taking the history/archeology course were compelled to critically analyze the subjects it covered. And, in fact, belief in those topics dropped by the end of the semester and dropped even more dramatically in the honors version of the course. More significantly, the students also showed lower levels of belief in topics that weren't covered in the class. Honors students again showed a larger effect. It's not clear whether the students in the honors class benefitted from the project portion of the class or whether the larger effect came because they were simply higher achievers.

It's not especially surprising that teaching critical analysis in the context of topics that have maintained the public's interest for decades would be more effective than doing the same in a specialized class for science majors. But it's important to remember that this was a small study, and it would be good to repeat it over several years at different universities and perhaps with other control classes. Still, the authors point out that these students had lower levels of uncritical belief than the population at large, and yet levels dropped even further after the class. So, there's a chance we'd see an even stronger impact on the general student population.

But professors might want to be cautious about doing this too widely, as there's some indication that it turns the students into real pains. After completing the course, one told the instructors: "I find myself challenging what my professors say when they use false dichotomies, circular arguments, or reference questionable sources."

Science and Education, 2017. DOI: 10.1007/s11191-017-9878-2 (About DOIs).