Medicine Gets Harder

Medicine is not easy. There is the huge amount of information you have to learn just to get through medical school. After graduation the information expands exponentially while my brain can only access it linearly. I was always jealous of Neo and wish I had a port at the base of my brain so I could access the Matrix,or better yet, PubMed, directly. I don't look good in a long leather coat anyway.

I read an enormous amount of the primary literature, but I am a clinician, not a scientist or a statistician. I have to rely upon the expertise of others, the notorious peer review, to vet papers. And even then I recognize that most published research findings are false. So what is a poor doctor to do? At least as a specialist I have somewhat of a mastery of my specialty and have an extensive background in ID to help put studies in perspective as they come out. Like much of my understanding of the world, I rely on what the preponderance of the ID literature suggests about a given treatment. The rest of medicine? I have trust the expertise of others.

There are now two further problems with literature.

First is that negative studies are neither written up nor published:

Of all the null studies, just 20% had appeared in a journal, and 65% had not even been written up. By contrast, roughly 60% of studies with strong results had been published.

That is an amazing percentage of negative studies that have never seen the light of day. Part of the reason we need a Journal of Negative Results, to prevent people from repeating negative studies and to prevent the literature from being biased by an excess of positive studies.

While the above study was in the social sciences, there is no reason to suspect that the magnitude of the issue is any different in the world of medicine and pseudo-medicine and probably skews many a meta-analysis.

But it gets even worse. This month in JAMA is Reanalyses of randomized clinical trial data.. For most studies the data is analyzed once, by the authors who have access to the data. They managed to find 37 studies that were reanalyzed and 35% of the time there was a different interpretation as a result. Which was the ‘real’ result, the original or the reanalysis? Looking at the studies, there was only one I would say the reanalysis resulted in the likely real result: a homeopathic treatment was effective in the original analysis and no better than placebo in the reanalysis. Given that homeopathy is placebo, one would except it to be equal to placebo. But the other studies? I can’t say.

As the authors point out, there are often multiple other studies on a similar intervention that help inform decisions by clinicians. A single study is rarely definitive. They also point out that

authors completing the reanalysis should not have conflicting financial, ideological or political interests.

a concept evidently ignored by Translational Neurodegeneration.

It makes reading and understanding the medical literature even more problematic. It may be wrong, the negative studies may be moldering on hard drive and the data analysis may be wrong. Or not.

I am again envious of pseudo-medical providers: they never have to worry about these issues. Ever. The certainty that comes with being reality-proof and the ability to just make stuff up.