Hi there, sorry to be absent (dayjob!). I was surprised to see a study I’m a co-author on getting some front page media play today, under the headline “Statins ‘have no side effects'”. That’s not what our paper found. But it was an interesting piece of work, with an odd result, looking at side effects in randomised trials of statins: specifically, and unusually, it compares the reports of side effects among people on statins in trials, against the reports of side effects from trial participants who were only getting a dummy placebo sugar pill.

Remarkably, people report typical statin side effects even when they are only receiving a placebo: the phenomenon of people getting unpleasant symptoms simply because they expect to is fairly well-documented, and it’s called the nocebo effect, the evil twin of the placebo effect. Here’s a piece I wrote a piece on the nocebo effect a while ago, specifically reviewing some of the earlier studies where people report side effects even when they’re only getting a placebo in a trial. It’s remarkable:

Can a sugar pill have a side effect? Interestingly, a paper published in the journal Pain next month looks at just this issue. They found every single placebo-controlled trial ever conducted on a migraine drug, and looked at the side effects reported by the people in the control group, who received a dummy “placebo” sugar pill instead of the real drug. Not only were these side effects common, they were also similar to the side effects of whatever drug the patients thought they might be getting: patients getting placebo instead of anticonvulsants, for example, reported memory difficulties, sleepiness, and loss of appetite, while patients getting placebo instead of painkillers got digestive problems, which themselves are commonly caused by painkillers. This is nothing new. A study in 2006 sat 75 people in front of a rotating drum to make them feel nauseous, and gave them a placebo sugar pill. 25 were told it was a drug that would make the nausea worse: their nausea was worse, and they also exhibited more gastric tachyarrhythmia, the abnormal stomach activity that frequently accompanies nausea. A paper in 2004 took 600 patients from 3 different specialist drug allergy clinics and gave them either the drug that was causing their adverse reactions, or a dummy pill with no ingredients: 27% of the patients experienced side effects such as itching, malaise and headache from the placebo dummy pill. And a classic paper from 1987 looked at the impact of listing side effects on the form which patients sign to give consent to treatment. This was a large placebo-controlled trial comparing aspirin against placebo, conducted in three different centres. In two of them, the consent form contained a statement outlining various gastrointestinal side effects, and in these centres there was a sixfold increase in the number of people reporting such symptoms and dropping out of the trial, compared with the one centre that did not list such side effects in the form.

Now, this has real world implications. If we tell people about side effects, and in doing so, we induce these unpleasant symptoms, then we are inflicting harm on our patients. Inflicting harm isn’t so unusual in medicine, in the process of doing good, but we aim to ensure that overall we do more good than harm, and in particular we aim to produce and share good quality information, so that patients can make informed decisions about the treatments they take.

With that in mind, we have a responsibility to try and establish good quality evidence on side effects, and in particular to nail down how far these side effects are genuinely being caused by the drugs. We certainly shouldn’t give false reassurance; but we also shouldn’t scare people into experiencing side effects; or scare them into avoiding a medication which might help them. (Some people get a bit melodramatic about statins, as if they’re being forced down our throats: the evidence shows they reduce your risk a bit if you’re at high risk of a heart attack; they’re less helpful – but still a bit helpful – if you’re low risk; and if you decide you don’t want to take them, after being appraised of the evidence, well, that’s easy, don’t take them).

As I explain in Bad Pharma, we are generally pretty imperfect at monitoring side effects, partly because it’s a hard job to do, and partly because there’s still a lot of dismal secrecy around: the WHO Uppsala side effects monitoring centre withholding information from researchers is a particularly disappointing example of this, as is the European Medicines Agency’s silly and rather self-defeating secrecy around the content of full Risk Management Plans.

And that brings me to the central flaw in our study. As we say in the text, the side effects information we were able to work with, from trial publications, is likely to be incomplete: the trial reports varied in what side effects they reported, they often failed to describe their methods for spotting and reporting side effects very well, and companies may not be highly motivated to put a lot of side effects data into their academic papers (to say the least).

Since the last draft of the paper (time moves slowly in academic publishing…) our knowledge of these flaws has deepened. I wrote in Bad Pharma about how side effects information can be buried, and the importance of access to something called the Clinical Study Report about a trial: these are very long and detailed documents that give a huge amount of detail about the methods and results of a trial, and they’re important, because methodological flaws can often be glossed over in the brief report on a clinical trial that appears as an academic journal paper. This is why asking for CSRs to be shared is one of the key asks of the AllTrials campaign, which I co-founded last year.

In a recent paper, we got a much clearer picture of how much information is missing: researchers from IQWiG (the German equivalent of NICE, but more muscular) compared CSRs against academic papers, side by side, and worked out exactly how much was missing from the journal publications. They found that CSRs consistently report much more complete information on methods and results. Table 3 is the money shot, most easily seen in the PDF: the amount of missing information on side effects in journal reports is particularly bad.

When I saw that the statins paper was finally coming out this week I tried to make an amendment, in amongst the many caveats in our discussion section (trial participants are often unrepresentative of everyday patients, as explained in Bad Pharma, etc…) but sadly I was too late. Here’s the small addition I wanted to make (in bold):

Comparison with real-life clinical experience Many real-world patients report muscle-related symptoms with statins. This contrasts with the low placebo subtracted rate in blinded trials shown in this meta- analysis. Several explanations are possible. First, commercial sponsors of clinical trials may not be motivated to search exhaustively for potential side effects. One pointer towards this is that, although liver transaminase elevation was documented in the majority of trials, new diagnosis of diabetes was only documented in three of the 29 trials. It is also likely that side effects data is collected, but not reported in the academic paper: a recent study by IQWiG, the German government’s cost effectiveness agency, found complete information for 87% of adverse event outcomes in the standard lengthy regulatory document for industry trials (the Clinical Study Report) but for only 26% of adverse event outcomes in the journal publication [Wieseler 2014]. Second, many trials do not state clearly how and how often adverse effects were assessed…. Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervölgyi V, Kohlepp P, et al. Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data. PLoS Med. 2013 Oct 8;10(10):e1001526.

That certainly doesn’t mean I think our paper is wrong. I think it’s a useful illustration of how we could – and should – gather side effects data from trials, and use this alongside other sources of imperfect information. This is especially true for commonly prescribed treatments like statins, because normally trials are too small to spot side effects, whereas here we have large enough numbers of participants in the trials, and a good chance of detecting and documenting rarer adverse events. Lastly, trial participants are subject to a very high level of scrutiny, so it’s a colossal missed opportunity if we fail to exploit that and document side effects as well as benefits.

So, overall, I think our paper uses the right method, on an important question, but our data was flawed.

And there’s an easy way to fix that. I’d like to repeat the study, using the CSRs on the trials as the source data on side effects, rather than the academic journal papers. That is a big piece of work, because companies generally refuse to share CSRs (although GSK have promised to, in signing up to AllTrials), while some like Abbvie and InterMune even sue regulators to keep them secret. Then, once you’ve finally managed to obtain these documents, they are huge and unwieldy, as the Cochrane group who’ve gone through the Tamiflu ones can attest.

But that would be the way to get a proper answer, and it would also have the interesting side effect of showing whether side effects really are obfuscated, in the editing process that happens between a lengthy and complete (but inaccessible) CSR, and a brief academic journal publication for doctors and researchers to read. If there was a big difference, that, I think, would be big potatoes.

If anybody wants to fund that, or has a year of a full time researcher to donate, I’m ben@badscience.net, please get in touch.

Lastly, though, if the findings from our paper are correct – and I’m fairly certain that muscle ache on statins, for example, often is a product of the nocebo effect – then this is yet another illustration of the remarkable power of the mind over the body.

There’s a ton of content about the placebo effect on this site (including a couple of BBC documentaries I did, all here). Lastly, after all that data and talk of ICH-GCP compliant Clinical Study Reports, here’s a video of me shouting on stage at the Hammersmith Apollo about the nocebo effect. See you soon!