My opinion about medical marijuana has been fairly consistent. First, the claims made by its advocates for it far exceed the evidence for its benefit, which is why I've referred to it as the "new herbalism." Of course, it's not really very new, but it is herbalism in that medical marijuana advocates make grandiose claims for using their favorite "drug" in its plant form rather than doing the standard thing that modern medicine does with natural products and try to isolate the active compounds, in this case a class of molecules known as cannabinoids. Second, although medical marijuana might have some minor utility in relieving the symptoms of chemotherapy in cancer patients, contrary to the claims of people like Rick Simpson promoting hemp oil as a cancer cure, his believers who provide anecdotes, and a large number of advocates who believe it is the next big thing in treating cancer, cannabis does not cure cancer. The bottom line: Purified cannabinoids have some promise for medicinal uses, but medical marijuana itself has little evidence to support its use and serves mainly as a politically palatable "foot in the door" for advocates to get their favorite drug legalized, and I say this as someone who thinks that marijuana should be legalized for recreational use.

I hate to say, "I told you so," but I told you so. No, wait. I love to say "I told you so," at least when the evidence is on my side. This time, it comes in the form of a systematic review and meta-analysis hot off the presses yesterday in JAMA looking at medical marijuana. Not surprisingly, the story's made the national news, including NPR, New Scientist, and Reuters. Amusingly, the headlines and emphasis varied rather widely. For instance, NPR's story was entitled Review Raises Troubling Question About Marijuana's Safety, Effectiveness while livescience reported Medical Marijuana: Review Shows Pot Helps These Conditions. This latter headline is known as looking at the glass as half full (possibly through a haze of pot smoke) while the former was perhaps a more accurate statement of the studies findings. Even The Cannabist produced a headline Study: Medical pot unproven for many illnesses OK’d by state law.

Let's dig in.

The study is by Penny Whiting and colleagues at the University of Bristol and entitled (appropriately enough) Cannabinoids for Medical Use: A Systematic Review and Meta-analysis. Let's just say that Whiting et al failed to find much in the way of compelling evidence supporting the use of medical marijuana for most indications, and, even for the indications for which there was positive evidence, it wasn't the sort of evidence that would exactly blow most doctors away. Overall, they examined 79 randomized clinical trials found through the following schema using the guidance published by the Centre for Reviews and Dissemination and the Cochrane Collaboration looking at either medical marijuana or various cannabinoids, such as dronabinol and nabilone, for a variety of indications. These trials compared cannabinoids with either usual care, placebo, or no treatment for the following indications: nausea and vomiting due to chemotherapy, appetite stimulation in HIV/AIDS, chronic pain, spasticity due to multiple sclerosis (MS) or paraplegia, depression, anxiety disorder, sleep disorder, psychosis, intraocular pressure in glaucoma, or Tourette syndrome. These indications were prespecified. If no RCTs were available for a particular indication or outcome (eg, long-term AEs such as cancer, psychosis, depression, or suicide), nonrandomized studies including uncontrolled studies (such as case series) with at least 25 patients were eligible.

If you want to get an idea of just how crappy the evidence base is, take a look at the description of the methods and characterization of the studies:

The searches identified 23,754 hits (records) of which 505 were considered potentially relevant, based on title and abstract screening, and obtained as full-text studies. A total of 79 studies (6462 participants), available as 151 reports, were included; 3 studies (6 reports) were included in multiple indication categories (Figure 1). Thirty-four studies were parallel-group trials (4436 participants), and 45 were crossover trials (2026 participants). Four studies were available only as an abstract, a further 3 were available only as abstracts but with additional details available on trial registries including full results in one, and details of 2 trials (including full trial results) were available only as trial registry entries; all other trials were reported in full-length journal articles. Where reported, the proportion of participants who were men ranged from 0% to 100% (median, 50% [57 studies]), and the proportion of white participants ranged from 50% to 99% (median, 78% [18 studies]). Publication dates ranged from 1975 to 2015 (median, 2004 [with one-third of trials published before 1990]). Studies were conducted in a wide range of countries. A variety of cannabinoids were evaluated and compared with various different active comparators or placebos; most active comparators were included in the nausea and vomiting indication ... Four (5%) trials were judged at low risk of bias, 55 (70%) were judged at high risk of bias, and 20 (25%) at unclear risk of bias (eAppendix 13 in Supplement 2). The major potential source of bias in the trials was incomplete outcome data. More than 50% of trials reported substantial withdrawals and did not adequately account for this in the analysis. Selective outcome reporting was a potential risk of bias in 16% of trials. These studies did not report data for all outcomes specified in the trial register, protocol, or methods section or changed the primary outcome from that which was prespecified. Most studies reported being double blinded but only 57% reported that appropriate methods had been used for participant blinding and only 24% reported that outcome assessors had been appropriately blinded.

In other words, the vast majority of the trials were not RCTs of sufficient rigor to be included in the meta-analysis. Of course, this is typical for most meta-analyses, which often start with hundreds of studies and end up with a few dozen at most. However, in this case, only a tiny minority of studies were sufficiently well-designed to be at low risk for bias, and over 2/3 of the studies were at high risk of bias for a wide variety of reasons, including sloppy double blinding (a huge problem). Just as bad are selective outcome reporting and potentially even worse is changing the primary outcome from the prespecified outcome, which is all too often used to "massage" the data to produce a positive result when the result for the prespecified primary outcome turns out negative.

There's a lot of information in the meta-analysis, but it can be fairly quickly summarized thusly. The authors basically looked at various indications for cannabinoids and then rated them according to GRADE (Grading of Recommendations Assessment, Development and Evaluation) to evaluate the overall quality of the evidence for risk of bias, publication bias, imprecision, inconsistency, indirectness, and magnitude of effect. GRADE ratings include very low–, low-, moderate-, or high-quality evidence reflect the extent to which the authors are confident that the effect estimates are correct. Interventions were further subdivided into different cannabinoids tested. Most of the studies examined isolated cannabinoids.

Overall, most studies showed some improvement of symptoms with cannabinoid use compared to control groups, but most didn't reach statistical significance and the effect sizes tended to be small to moderate. For example, here are some of the findings:

Nausea and vomiting due to chemotherapy: OR 95% confidence level, 3.82 (1.55 to 9.42) for nausea and vomiting, complete response; GRADE: Low. HIV/AIDS weight gain: OR 95% confidence level, 2.2 (0.68 to 7.27) for greater than 2 kg weight gain over six weeks; GRADE: Low. Chronic pain (neuropathic and cancer): Low to moderate evidence of efficacy by GRADE. Spasticity due to multiple sclerosis or paraplegia: Low to moderate evidence of efficacy by GRADE. Depression: Evidence favors placebo, but GRADE is very low. Anxiety disorder: Evidence favors cannabinoids, but GRADE is very low. Psychosis Evidence favors cannabinoids, but GRADE is low. Tourette's syndrome: Evidence favors cannabinoids, but GRADE is low.

Just to give you an idea, here is a graphical summary of the effect of cannabinoids on pain. If the error bars overlap zero, then the effect is not statistically significant.



Odds indicate 30% or greater improvement in pain with cannabinoid compared with placebo, stratified according to cannabinoid. The square data markers indicate odds ratios (ORs) from primary studies, with sizes reflecting the statistical weight of the study using random-effects meta-analysis. The horizontal lines indicate 95% CIs. The blue diamond data markers represent the subtotal and overall OR and 95% CI. The vertical dashed line shows the summary effect estimate, the dotted shows the line of no effect (OR = 1). Improvement in PainOdds indicate 30% or greater improvement in pain with cannabinoid compared with placebo, stratified according to cannabinoid. The square data markers indicate odds ratios (ORs) from primary studies, with sizes reflecting the statistical weight of the study using random-effects meta-analysis. The horizontal lines indicate 95% CIs. The blue diamond data markers represent the subtotal and overall OR and 95% CI. The vertical dashed line shows the summary effect estimate, the dotted shows the line of no effect (OR = 1).

In other words, the evidence is reasonable, but by no means a slam dunk, given the methodological difficulties with the studies.

It was also noted that, like all drugs, cannabinoids have a risk of adverse events (AE):

There was an increased risk of short-term AEs with cannabinoid use, including serious AEs. Common AEs included asthenia, balance problems, confusion, dizziness, disorientation, diarrhea, euphoria, drowsiness, dry mouth, fatigue, hallucination, nausea, somnolence, and vomiting. There was no clear evidence for a difference in association (either beneficial or harmful) based on type of cannabinoids or mode of administration. Only 2 studies evaluated cannabis.59,77 There was no evidence that the effects of cannabis differed from other cannabinoids.

Again, cannabinoids, be they isolated and purified by a pharmaceutical company or as part of a burned plant being inhaled, are drugs and can thus cause adverse events. There is nothing magical about cannabinoids or medical marijuana, but, all too frequently, in the heartbreaking anecdotes used to support the legalization of medical marijuana, the weed (or the oil extract thereof) is portrayed as a miracle cure. Witness the case of Stefanie LaRue, for example, or the case of Charlotte Figi.

In an accompanying editorial, Deepak Cyril D'Souza and Mohini Ranganathan ask if medical marijuana is "putting the cart before the horse." They point out that for most of the indications that states are legalizing medical marijuana for there is little or no evidence that cannabinoids are efficacious in relieving symptoms and virtually no high quality evidence. In addition, they observe that the FDA requires evidence from at least two well-designed randomized clinical trials before it will approve a drug for a given indication and that for the vast majority of the conditions for which states have legalized medical marijuana the evidence doesn't come close ot meeting that standard, being mostly anecdotal.

Indeed, to that I'd add that usually it's seemingly compelling testimonials, far more than anything, that drive legislators' approval of medical marijuana. Another problem is that, unlike most FDA-approved drugs, whose active components are well-characterized and few in number (usually one), marijuana is "a complex of more than 400 compounds including flavonoids and terpenoids and approximately 70 cannabinoids other than Δ9-tetrahydrocannabinol (THC)," that "results of studies with individual cannabinoids (eg, THC or CBD) cannot be extrapolated to marijuana and vice versa," and that "unlike FDA-approved medications that have a relatively uniform composition, the composition of cannabis preparations can vary substantially in its content of THC and CBD, such that precise dosing may be difficult." As I said, it's the same problem that herbalism has compared to purified natural products. D'Souza and Ranganathan rightly wonder why there is a double standard, with medical marijuana being held to a much lower (practically nonexistent) standard of evidence. If that standard of evidence that would provoke howls of outrage—and rightly so—if applied to approval of drugs produced by big pharma.

None of this is to say that there might not be indications for which cannabinoids will be safe and efficacious. Arguably, this meta-analysis suggests two of them, chronic pain and spasticity, and as I've discussed before there is some evidence for epilepsy. However, as is the case with much advocacy-based, rather than science-based medicine, the claims far outweigh the actual scientific and medical promise. Then, as D'Souza and Ranganathan note, there's this:

In conclusion, if the states’ initiative to legalize medical marijuana is merely a veiled step toward allowing access to recreational marijuana, then the medical community should be left out of the process, and instead marijuana should be decriminalized. Conversely, if the goal is to make marijuana available for medical purposes, then it is unclear why the approval process should be different from that used for other medications.

Which is what I've been saying all along. If pot should be legalized, just legalize it, and stop the charade that it's all for medical indications.

Coincidentally, the other day I was involved in a discussion where a graduate student described a lecture in ethics and law she attended in which the professor said that the law always follows the science. That's not true at all. The law frequently puts the cart before the horse, as it has done in many states with mandatory breast density reporting laws and as it is doing with medical marijuana laws, both cases where the law has gone far beyond what the science can support. Unfortunately, although D'Souza and Ranganathan recommend putting the horse back in front of the cart, once the law goes beyond the science it's rare that it can be put back. That's because, more often than not, in politics a heart-rending anecdote will trump cold science.