This morning I followed a Google Alert to an article in Medical News Today, which reviews the results of a new meta-analysis exploring the long-term outcomes of taking antipsychotic medication. The article confidently claims: “Lieberman and team looked at clinical trials and neuroscientific data, and they found that the therapeutic benefits of antipsychotic medication far outweigh their side effects.” The journalist, Ana Sandoiu, goes on to quote Lieberman as stating that “Anyone who doubts this conclusion should talk with people whose symptoms have been relieved by treatment and literally given back their lives.“

For my doctorate research, I talked with 144 people who take or have taken antipsychotics and a third gave me these kinds of descriptions. Another third said quite the opposite, and I can hear them yelling at me to share their side of the story.

There is no citation or web link to the original paper so it is difficult to verify whether Sandoiu’s interpretation of the results or use of quotations is accurate to the researchers’ intent. But I managed to track down the abstract (Goff, Falkai, Fleischhacker, et al, 2017, in press ), which states: “Little evidence was found to support a negative long-term effect of initial or maintenance antipsychotic treatment on outcomes, compared with withholding treatment. Randomized controlled trials strongly support the efficacy of antipsychotics for the acute treatment of psychosis and prevention of relapse.”

This doesn’t actually say they found that the benefits outweigh the costs. It says they did not find a negative long-term effect. It says they did find a positive effect on symptom reduction in the short-term. The grand claim may seem to be the obvious implication, but we cannot assume that statistical significance always translates into personal significance, and there are other relevant measures of recovery missing from the analysis so the weighing-up process is flawed. The subtleties have definitely been lost in translation. Instead we are offered an oversimplified shot at absolute truth that totally obscures individual variation, and invalidates the perspectives of those who report negative effects — the very people who tend to drop out of clinical trials and be lost to long-term follow-up.

The perspectives of people who take antipsychotic medication weren’t considered in this meta-analysis at all. Nor was the possibility that unmeasured psycho-social factors might account for the variation in outcomes these studies observed. People who take antipsychotic medications do not spend their lives sitting inside a vacuum passively receiving their experiences from chemicals in their brains. Social support, coping style, occupational activities and situational stressors are just as relevant to the outcomes of people who take antipsychotics as they are to any other group of people. If the research hasn’t controlled for these factors, and the vast majority has not, there is no way we can confidently declare that the benefits of antipsychotics universally outweigh the costs. From what I can see, the researchers did not actually make such a sweeping claim, but no one reading Sandoiu’s article will know this.

I’ve spent the last five years immersed in the antipsychotic research. As a whole this is a body of literature that seems obsessed with statistically proving the rightness or wrongness of taking antipsychotic medication. For every few studies that report a benefit to persisting with antipsychotics (Alonso et al., 2009 ; Haro, Novick, Perrin, Bertsch, & Knapp, 2014 ), there is an article that finds a long-term cost, or no difference at all (Harrow, Jobe, & Faull, 2012 ; Landolt et al., 2016 ; Wils et al., 2017 ; Wunderink, Nieboer, Wiersma, Sytema, & Nienhuis, 2013 ).

It is possible that each of the above is true for different people at different times, but the meta-analysis simply weighs in on the existing good-bad/right-wrong debate. It was likely predestined to reveal a benefit because problems with publication bias mean there are more studies with positive results than negative results to go into the meta-analytic pot. We need to shift towards looking at the within-group variation among those who persist long-term and among those who discontinue — this is where we will discover how to improve the recovery outcomes of both sets of people.

One of the major insights I took from the 144 people who answered my survey was that individual experiences vary and they are all valid. In my study, overall subjective experiences ranged on a continuum from “life-saver” to “hell” and every point between (Larsen-Barr, 2016 ). Around a third reported overall positive experiences such as “A major relief from the monsters […] for me they have saved my life” and “Helped me get through an unstable period of my life.” And around a third of the participants reported mixed experiences such as, “A short term help when needed then a burden” and “A double edged sword. They help me with my bad experiences but they also take away the wind in my sails.”

Another third reported wholly negative experiences such as, “The worst experience of my life […] affected every aspect of my health and wellbeing.” The therapeutic benefits certainly did not outweigh the costs for those who described the overall experience of taking antipsychotics as “The ruin of my life” or said they were “Helpful to a point but […] robbed me of everything I value in myself as a person.”

The claim that the benefits of antipsychotic medications conclusively outweigh the adverse effects just is not true for some people. It is true for others, and for some people it is true in the short-term and later stops being true. I look forward to the day we can stop debating which group of people really exists because they clearly all do. The only way to determine whether the benefits are outweighing the costs is to ask the individual experiencing them, and to keep asking over time. A quantitative meta-analysis cannot give us the answer.