If this is “evidence based medicine” I want my old job back

The news that NICE has put acupuncture and chiropractic on the list of approved therapies for non-specific lower back pain has led to about the reactions you’d expect – back-slapping and high-fiving from the crystals and “life force” crowd, agonised complaining from the professional skeptics. But it’s actually a sign of something that ought to make us worry, not much but at least a little bit, about the way in which we’re doing medical science in this country.



The National Institute for Clinical Excellence is one of the world’s greatest temples of “evidence-based medicine” – the doctrine that as much medical practice as possible ought to be carried out using proven algorithms based on empirically valid evidence from controlled scientific experiments, rather than individual clinical judgement. It’s quite a controversial topic in the medical literature. While on the face of it, it sounds like obviously the right thing to do – after all, who doesn’t want to be “evidence based”? – it has attracted its share of zealots and nuts, and provoked something of a reaction from the part of the medical profession that believes that medicine is still as much of an art as a science.

And so, NICE has decided, on the basis of “the evidence”, that acupuncture and chiropractic are a good way to spend the NHS’s money. Except when you look at it, “the evidence” isn’t really all that great. As Edzard Ernst points out, the Cochrane Institute (the other great temple of evidence-based medicine) actually found chiropractic to be more or less useless, while the evidence for acupuncture is that all of the ancient wisdom and theory of the meridians and qi doesn’t actually confer any great benefit over and above that which can be gained from simply lying on a table and being poked with sticks.

Part of the problem is that in the specific case of lower back pain, it’s a notoriously difficult condition to understand or treat, and a lot of the art is simply to find a nice and professional-sounding way of saying “live with it, there’s nothing we can do” that doesn’t make the patient give up hope and suffer even more. But another part of the problem is that the overall assessment of what “the evidence” was, was made by a committee that had a bunch of spinal manipulation enthusiasts on it.

Which brings me to the problem; this is exactly what we should have expected, and it’s the reason why I’ve been putting the phrase “the evidence” in great big scare-quotes. Because the actual medical evidence on lower back pain isn’t something that can be nicely summarised in a slim paper guideline; it’s spread out across millions of individual lower backs, some fraction of the experiences of which are summarised into hundreds of research papers, which were then distilled down into the Cochrane review, which was itself processed through the NICE committee. Basically what evidence-based medicine is about, at this level, is somebody making a decision about what the facts are going to be.

And if that decision about “what the facts are” is one that is going to determine the handing out of large chunks of government cash, then you bet that the enthusiasts of every theory there is are going to move hell and high water to get themselves on that committee. Not out of any venial motive, but because they believe in their theory, and a contrary NICE guideline has the potential to kill it stone dead. So what happens is that the process of finding out the underlying truth, which is of necessity slow, unclear and often completely open-ended, gets accelerated and politicised. It’s what you might call “government science”.

It’s a phenomenon that’s very familiar to economists under the name “Goodhart’s Law”. Basically, Goodhart’s Law says that “any economic relationship which is used for policy purposes, ceases to be valid”. In other words, you can have an economic model which works tolerably well as an understanding of how, say, the relationship between money, prices and output works. But when you try to use that model to set interest rates, then suddenly the model itself is part of the recursion – part of the system that you’re trying to control – and this changes the nature of the relationship that you were trying to use.

Similarly, in the early days of the evidence-based medicine movement, when they were the Young Turks or punk rockers, shaking up a complacent medical establishment that had got out of touch with the cutting edge of medical research, they had the potential to do a lot of good. But now they are the establishment, and as a result of that, the very evidence that they rely on, is shaped by the fact that it needs to appeal to them. The fact that a movement which begun by trying to bring science back into medicine, has now ended up putting its imprimateur on some obvious pseudoscience, ought to worry us more than it does, because this is only the most obvious manifestation of the general problem.

After all, pharmaceuticals companies produce an awful lot of science, and they are very much interested parties to anything which might influence the spending of a lot of money on medicine. And we know from the Elsevier scandal that we can’t assume that the published scientific literature is immune. Evidence based medicine is an important part of modern science, and nobody wants to go back to the bad old days, but if the modern medical establishment doesn’t start getting a bit more sophisticated about sociology of knowledge and public choice economics, then the “evidence” that drives our medical treatment is going to end up being manipulated like a jump jockey’s lower back, and with similarly painful results.