Food science has a huge statistics problem. The solution, for now? Stop treating new nutrition studies like they contain the truth. Part three of three.

There’s a reason everyone’s confused about whether coffee causes cancer, or whether butter’s good for you or bad. Food research has some big problems, as we’ve discussed here and here: questionable data, untrustworthy results, and pervasive bias (and not just on the part of Big Food). There’s reason to hope that scientists and academic journals will clean up their acts, and that journalists will refine their bullshit detectors and stop writing breathlessly about new nutrition “discoveries” that are anything but. Until that happens, though, we all need to get better at filtering for ourselves.

A pair of recent articles coming out of the statistical community offers a terrific tool for doing just that—not a long-term fix, but a little bit of much-needed protection while we wait for something better. To understand it, though, we’re going to have to dip our toes into some chilly mathematical waters. Stick with me. It won’t be too bad.

Let’s look at three recent reports of scientific findings about diet:

Fifty grams of prunes a day prevents the loss of bone mineral density in elderly women with osteopenia Forty-eight grams of dark chocolate modulates your brainwaves for the better. Feeding infants puréed pork causes them to put on more body length than feeding them dairy.

They’ve all been peer-reviewed. All the findings have been declared to be statistically significant. And they all imply a clear cause-and-effect between a common food and a health outcome. And yet we know that there’s a good chance that at least one of them—and maybe even all three—will subsequently be proven to be false. So which ones does it make the most sense to ignore?

When two wrongs make a right

Here’s the problem with many of the nutrition studies you’re likely to read about in the press: Like most research, they’re carried out using an incredibly counterintuitive method called “null hypothesis testing.”

It goes like this. First, you start with whatever it is you’d like to prove—say, that drug X cures cancer. But then, instead of trying to prove your hypothesis directly, which is virtually impossible in the real world, you posit its opposite. For example: “I’m trying to prove that any connection between using drug X and curing cancer is just a matter of random chance.” That somewhat confounding non-statement is your null hypothesis.

Then you run your experiment and analyze your numbers. If you’re lucky, you’ll find that there’s not enough evidence to prove no connection between taking drug X and curing cancer. (Confusing, right?) Put another way, you’ve proven that the connection between drug X and cancer cures is not a matter of chance. Therefore, the thinking goes, drug X must cure cancer.