“We aren’t teaching students how to think critically!” So goes the exasperated lament you have probably heard and possibly uttered. The thing is, that’s a crazy hard thing to do. It may seem like a logic class should teach you to think in a more disciplined way, for example, but the sad fact is that those mental habits are very unlikely to transfer beyond the walls of the logic course. There are many different styles and contexts of critical thinking, and there is no magic subroutine that we could insert into our mental programming that covers them all.

But despair is not the only option. Effective coursework can build important and useful critical thinking skills. Doug Bonn at the University of British Columbia and Stanford’s N.G. Holmes and Carl Wieman focused on good scientific, quantitative thinking when teaching a group of first-year physics students. And like good critically thinking educators, they put their strategy to the test and published the results so they can be evaluated by others.

In this freshman calculus-based physics course, students worked through weekly experiments in lab sections as most physics students do. But the researchers tried a little something different a couple years ago when a fresh class of 130 students came in. In their early lab sections, the students were guided through comparisons between multiple experimental datasets and between experimental datasets and mathematical models.

By applying some statistics they were gradually learning, they grappled with why their comparisons came out the way they did. Rather than simply chalking up mismatches to “we’re just students, and our measurements probably aren’t perfect," as students often do, they considered modifying their experiments. How could they reduce their error bars? Were the data telling them the mathematical model was incorrect?

The idea was to foster the kind of thinking scientists use. With practice, the students should start to get an appreciation for interpreting real-world data and sometimes be confident enough to challenge models when they have high-quality data that demands it. To help, the students were given simplified decision trees—for a given statistical result, it directed them to the possibilities they needed to decide among.

Over the course of the year, these detailed critical analysis exercises were gradually phased out, leaving students to their own devices. The researchers chose three activities to analyze for student behavior: one from the second week and two from the end of the course, long after the explicit critical thinking instructions had disappeared. Looking over lab notes, they noted how many of the students contemplated modifying their experiments to get a better data and how many identified or interpreted mismatches between their data and the model.

For comparison, they used the previous year’s class, which had completed an identical course lacking only the new critical thinking instructions. (And yes, the two classes scored similarly on tests taken at the start of the year.)

They found that the experimental class did more of this critical evaluation early in the course, which is hardly a shock given that they were instructed to. However, they also did more at the end of the course. The habits had stuck with them. The improvement was significant, with the experimental class 12 times as likely to modify their experiment and four times as likely to challenge an incorrect model in their final activity. A little more qualitatively, the researchers write that the students “also showed much more sophisticated reasoning about their data.”

Interestingly, the researchers tracked the same students into the sophomore physics course that a third of the freshmen had advanced into. Even there, they still saw improvements, despite the fact that none of the critical thinking instructions were repeated in that course.

The researchers write, “The cycles of making and deciding how to act on quantitative comparisons gave students experience with making authentic scientific decisions about data and models […] With a focus on the quality of their data and how they could improve it, the students came to believe that they are able to test and evaluate models. This is not just an acquisition of skills; it is an attitudinal and epistemological shift unseen in the control group or in other studies of instructional laboratories.”

It’s not like this is a revolutionary concept—many educators try to build similar sorts of activities—but it’s nice to see measurable benefits. The researchers emphasize that the basic structure of their approach should work in pretty much any course, but that might be a tad optimistic. Not all sciences can provide tidy little quantitative experiments that students can easily iterate in a three-hour lab section (though there are ways to approximate that experience) or a neat mathematical model to connect them to. The statistics will also be an obstacle for some students.

And, of course, this is no critical thinking panacea. We don’t know what kinds of contexts beyond physics classrooms these habits may or may not translate to. That said, this is a pretty darn good critical thinking skill to impart—it gets to the heart of the scientific method.

PNAS, 2015. DOI: 10.1073/pnas.1505329112 (About DOIs).