It’s a #SkepticalTuesday on a Wednesday post. A #SkepticalWednesday, if you will.

***

So the Washington Post published this article the day before yesterday in their Health and Science section. It’s called “Acupuncture may help lower high blood pressure.” But if you have high blood pressure, maybe you should just stick to medicine we know works.

The TL;DR is: “error bars are important.”

The WaPo article cites a study in the journal Medical Acupuncture, which we’ve seen before in the skeptical community. All I’m going to say is that with a title like that, there’s a chance (just a chance, mind you) that we might be dealing with confirmation bias in this post.

So the article, which can be found here (probably behind a paywall, it’s hard to tell where I am), claims that “After 8 weeks, 33 patients treated with EA at PC 5-6 + ST 36-37 had decreased peak and average SBP and DBP, compared with 32 patients treated with EA at LI 6-7 + GB 37-39 control acupoints. Changes in blood pressures significantly differed between the two patient groups.” I’m here to tell you that no, they did not differ significantly. Because error bars.

The basics of the study are that they were using ElectroAcupuncture (that’s acupuncture with electricity added) on two groups of people with mild to moderate hypertension, who weren’t on drugs for it, and who were randomized into two groups for the first eight weeks of the study (I’m not going to go into the follow-up data here because I just don’t have the time. Maybe some other week.). The two groups were treated with what we’ll call “real” EA and “fake” EA: EA at two “real” sets of points (PC 5-6 / near the wrist, and ST 36-37 / just below the knee), and two “fake” sets of points (LI 6-7 / a little further up the forearm, and GB 37-39 / a little above the ankle). These are all “real” acupuncture points, they just aren’t supposed to treat high blood pressure (and I’ll take a moment to point out that if acupuncture is really effective, applying it wrong just to prove a point seems terribly reckless, but I digress).

After eight weeks of treatment, the paper claims, “EA at [the “real” points] decreased peak SBP and average SBP, DBP, and mean BP during the 8-week course of therapy […] EA treatment in 32 patients at [the “fake”] acupoints did not consistently decrease SBP and DBP or heart rate. Thus, in contrast to EA treatment at [the “fake” points], SBP, DBP, and mean BP were reduced after 8 weeks of EA applied to [the “real”] acupoints in hypertensive patients.”

Look, setting aside that they conveniently leave the supposed inconsistency of the “fake” treatment in relation to mean BP, the main problem is that they drew their graphs using the center of each error bar, instead of checking whether there was possible overlap within the margin for error. In the below three images, you can see the “real” point data on the left, and the “fake” on the right, for systolic, diastolic, and mean blood pressure of the study participants over time.

In red, you can see I’ve drawn lines within the bars that represent the margin for error that are either nearly identical or exhibit better outcomes, especially in the case of mean blood pressure.

These are extreme lines I’ve drawn, certainly, but that’s what error bars are for: to show you whether it’s possible for your results to be artifacts of things like, I don’t know, confirmation bias.

I’m not saying that zapping people in the right places with electricity won’t lower their blood pressure. (Hell, zap someone with enough electricity and I guarantee you their blood pressure will drop.) But it’s an extraordinary claim that requires a higher standard of proof than this. If the opposite result to the one you report is within the margin for error in your data, then at the very least you need a new dataset to prove your point. And if the difference between your treatment and no treatment is barely visible for a condition that we already have an effective treatment for, I don’t know what to tell you, except that it’s probably a waste of money, time, and effort. But, as always, your mileage may vary.

***

Richard Ford Burley is a writer, library worker, and doctoral candidate in English at Boston College, where he’s studying remix culture and the processes that generate texts. In his spare time he writes about science, skepticism, and feminism (and acupuncture claims) here at This Week In Tomorrow.