Deborah Cohen , associate editor, The BMJ dcohen{at}bmj.com

A study published in The BMJ this week shows how most new cancer drugs are failing to deliver any clinically meaningful benefit. It’s time for Europe to raise the evidence bar before market approval, finds Deborah Cohen

Most cancer drugs recently entering the European market do so without clear evidence of extending or improving quality of life, new research published in The BMJ has found.1

The findings raise serious questions about why the current regulatory environment supports the approval of cancer drugs that may leave patients at risk of experiencing toxicity and reduced quality of life without deriving meaningful benefit.

Out of the 68 cancer drug indications approved by the European Medicines Agency during 2009-13, 57% (39) entered the market without evidence of a survival or quality of life benefit. Even when drugs did show survival gains over available treatment options, most of these were not clinically meaningful, researchers found.

Many of the drugs were approved on the basis of surrogate endpoints despite evidence that these are not a reliable indicator of overall survival or quality of life for most cancer treatments.

“When expensive drugs that lack clinically meaningful benefits are approved and reimbursed within publicly funded healthcare systems, individual patients may be harmed, important resources wasted, and the delivery of equitable and affordable care is undermined,” the researchers say.

The study comes at a time when European governments are starting to seriously challenge the high cost of drugs. While it’s hard to know how much healthcare systems are paying for cancer drugs because prices are often negotiated behind closed doors, the total amount spent on cancer care is growing, partly because of the cost of drugs.

Inadequate evidence The research found that the EMA is basing many approval decisions on uncontrolled study designs or surrogate endpoints, which don’t always translate into outcomes which make a difference to patients. Some of the cancer drugs were given “conditional marketing authorisations,” on the understanding that postmarketing studies would assess overall survival or quality of life. If the drugs are subsequently found to be clinically ineffective or unsafe, then the EMA can withdraw them. The study in The BMJ identifies 10 drugs approved under these fast track arrangements, but after four years of market entry none of them had good evidence that they either extended or improved life for patients. Asked to comment on the research by The BMJ, an EMA spokesperson stated that is not agency practice to comment on research it has not seen. This is a topic that it has discussed widely and it welcomes taking part in further debate on the evidence underpinning cancer medicines. The fact that so many of the new drugs on the market lack good evidence that they improve patient outcomes puts governments in a difficult position when it comes to deciding which treatments to fund. In 2016, European health ministers issued a statement saying that new medical products “pose challenges to individual patients and public health systems in particular regarding their added value.” This, they said, affects patient access, affordability, and the financial sustainability of health systems. As an example of the pressures put on health systems, a recent Bristol-Myers Squibb funded analysis compared licensed drugs for six different cancers across Europe and Canada and concluded that reimbursement decisions seem inconsistent. In an accompanying press release, the authors of the analysis stated: “There are potentially 200 000 patients in 12 countries who by licence should have access to drugs but are not getting them because of the reimbursement decision.” Their underlying premise was that the EMA grants licences to “safe, effective cancer treatments where access to the drug can improve and prolong life” and these drugs should be paid for.2 In the eyes of industry the problem is with national reimbursement, yet the overwhelming picture now, from not only The BMJ study1 but from studies published in Lancet Oncology and elsewhere, is that cancer medicines are being licensed that do not deliver clinically meaningful benefit. The BMJ has found methodological problems with trials that EMA has either failed to identify or overlooked (box 1). This includes the trials’ design, conduct, analysis, and reporting. Box 1: EMA’S questionable practices Lenience on trial design EMA’s approval of mifamurtide in 2009 for non-metastatic osteosarcoma was on the basis of a pivotal trial that was not designed to evaluate the drug’s clinical efficacy. Instead, the factorial design trial was aimed at independently evaluating the effect of mifamurtide and another agent, ifosfamide. In 2007, the FDA’s oncologic drugs advisory committee raised serious concerns about the study design and choice of comparators and concluded that mifamurtide failed to show substantial evidence of efficacy

Pomalidomide was approved by the EMA in 2013 for relapsed or refractory multiple myeloma. The trial supporting this indication compared one group of patients who were randomly allocated to receive pomalidomide in combination with low dose dexamethasone to another group receiving high dose dexamethasone. The company was repeatedly alerted about the unsuitability of the comparator choice for regulatory decision making by the FDA and NICE Failure to follow its recommendations The main study supporting the EMA’s marketing authorisation of S-1 (tegafur/gimeracil/oteracil) in combination with cisplatin for the treatment of advanced gastric cancer was designed and conducted to show its superiority over 5-fluorouracil in combination with cisplatin. When the trial did not meet its objective, and after it was completed, the sponsor company considered it appropriate to change the primary objective from superiority to non-inferiority. EMA’s guidance from 1999 admitted that “there is ample room for bias” if the non-inferiority margin is chosen after the data have been inspected Tolerance for questionable analytical practices In 2011, EMA approved bevacizumab as a first line treatment for advanced ovarian cancer. Two pivotal studies supported this extension of bevacizumab’s previously approved indication. In one of these studies, the primary endpoint was initially specified as overall survival but was changed to progression-free survival during the trial, jeopardising a fundamental principle in trial design.

Vinflunine was approved by the EMA in 2009 as a second line treatment for metastatic transitional cell carcinoma of the urothelial tract. In its pivotal trial, vinflunine was associated with an overall survival benefit over best supportive care. However, this effect was observed only in the those treated per protocol and not in the intention-to-treat population

Trastuzumab received marketing authorisation from the EMA in 2011 for locally advanced breast cancer. The main study supporting this approval measured overall survival as a secondary endpoint. Although the predefined analysis did not show a statistically significant survival benefit, EMA concluded that “the strongest evidence of benefit was provided by overall survival results” on the basis of an exploratory analysis that achieved significance after data were excluded from one of the participating centres “for which issues were raised.” In the five year follow-up of the trial, investigators concluded that the difference between groups in overall survival did not reach significance, probably because of crossover to adjuvant trastuzumab in some patients initially randomised to chemotherapy alone

Panitumumab was approved by the EMA in 2011 as a second line treatment for colorectal cancer on the basis of a randomised controlled trial with the coprimary endpoints progression-free survival and overall survival. According to the predefined statistical analysis plan, progression-free survival and overall survival were analysed using a P value of 0.01 and 0.04, respectively.3 At the time of approval, primary analysis showed a borderline statistically significant benefit in progression-free survival, which the EMA did not consider robust. There was no statistically detectable overall survival benefit. However, a later publication in the postmarketing period reported that the final analysis of this trial showed significant improvement in progression-free survival.4 Yet the P value for this analysis was 0.023, which did not meet the investigators’ prespecified threshold. Such flawed clinical trials can lead to bias and further difficulties in identifying the true effectiveness. Unless there’s thorough scrutiny of this regulatory evidence after approval, governments may make poor decisions about how to prioritise health budgets.

Unrealistic expectations Perhaps most importantly, however, the fact the drugs have been given the imprimatur of regulatory approval may cause patients and doctors to have unrealistic expectations about their benefits and harms. According to Richard Sullivan, professor of cancer and global health at King’s College London and director of the Institute of Cancer Policy, doctors cannot be expected to be gatekeepers. In many cases across Europe new cancer medicines with low clinically meaningful benefit continue to be prescribed. “They may inappropriately script cancer drugs because of patient and family pressure; a lack of understanding of how new complex therapies work; or because of the culture of medical oncology in the absence of multidisciplinary decision making,” he says. “If patients are not offered alternative modalities, including palliative care, in end-of-life settings then the risk of inappropriate or futile treatment with chemotherapy and immunotherapy increases.” Sullivan says the processes that allow a drug to be funded in national health services across Europe vary in their robustness and due diligence around judging clinical evidence. Some health technology assessment bodies view themselves as secondary gatekeepers to stop use of drugs that the EMA has licensed without evidence of benefit (box 2). Box 2: EMA under scrutiny Some aspects of the EMA’s regulatory process are coming under scrutiny—not least the scientific advice offered companies seeking approval for their drugs. According to Guido Rasi, EMA’s executive director, early dialogue with medicine developers allows the agency to give scientific advice and help with protocols to “provide methodological direction and discourage the production of irrelevant or substandard data.”5 Currently, the scientific advice EMA gives is not publicly available, preventing its assessment. Some advocacy groups have reported that attempts to access such information have been thwarted by commercial confidentiality. People both currently and formerly involved with EMA have told The BMJ that manufacturers see presubmission processes as a way to lobby the agency, repeatedly asking the same question until they get the answer they want, and this may affect various aspects of trial design, conduct and analysis. In July this year Emily O' Reilly, the European Ombudsman, launched a “strategic inquiry” into EMA’s processes. Although she recognised that “these activities help the development and availability of high-quality, effective and acceptably safe medicines,” such “activities may pose some risks.” She noted that the EMA sees presubmission meetings as a way to “enable medicine developers to establish contact with the agency staff who will be involved with the application.”5 The case of cancer drugs also raises question about inconsistency between funding and licensing decisions in Europe. One suggestion has been for the EMA to work alongside the organisations carrying out health technology assessments, which countries use to help decide whether to pay for a treatment. On paper, joining up the EMA with health technology assessment might seem a quicker way of getting drugs to patients. But concerns are being raised that EMA standards might actually erode those applied by some health technology bodies. More importantly, perhaps, is that decisions about pricing and reimbursement should also be related to the gross domestic product of countries, which differs greatly across the EU. Differential pricing would not be possible if assessment was linked with EMA approval. Furthermore, members of some health technology bodies say they are currently a barrier against poor regulatory decisions and worry that EMA’s “regulatory capture”—whereby 89% of the agency’s budget comes from the drug industry fees—may come to thwart their independence.6 For example, the EMA approved vinflunine as a second line treatment for metastatic transitional cell carcinoma of the urothelial tract in 2009 (box 1) on the basis of a potentially biased analysis. But the UK National Institute for Health and Care Excellence (NICE) was less convinced, implying the evidence used for regulatory approval did not show the drug to be effective, and didn’t recommend it. Although the exact reasons for rejection aren’t always clear, an assessment body analysis obtained by The BMJ shows that health technology assessments in most European countries have also taken a less favourable view of vinflunine than the EMA. For patients, approval of such drugs may lead to unrealistic expectations, fuelled by patient organisations . In response to NICE’s decision, Action on Bladder Cancer, a charity supporting patients and promoting research, wrote to the agency to complain that: “Patients with metastatic bladder cancer are disadvantaged by the lack of a second line treatment option. Study 302 is the first trial to show a survival benefit and we feel that vinflunine should be available for this relatively small group of patients.”7 Uncertainty is compounded by unproved drugs being used as comparators. Despite the questions around vinflunine , for example, it is now being used as a comparator in trials for new drugs. On its website, the patient charity Fight Bladder Cancer has highlighted an ongoing study of a drug called PDL3280A for patients with advanced or metastatic bladder cancer.8 This, they say, compares chemotherapy with either paclitaxel or vinflunine. But regulatory sanctioning of a comparator that lacks robust evidence of efficacy, means the cycle of weak evidence and uncertainty continues.

No one wants to say no to a cancer drug Vinflunine isn’t an isolated example of questionable decision making. In 2011, the EMA licensed panitumumab in combination with other drugs as a second line treatment for colorectal cancer. This was despite the agency questioning whether the primary analysis showing a borderline statistically significant benefit in progression-free survival was robust (box 1). Indeed, the EMA initially rejected the drug but later reversed its opinion.9 In the words of one EMA adviser who spoke to The BMJ, however, “no one wants to say no to a cancer drug.” When NICE invited Amgen, panitumumab’s manufacturer, to submit evidence for approval to use on the NHS, the company declined to do so. They intimated that there wasn’t sufficient evidence to determine the cost effectiveness. Again NICE didn’t recommend it.10 Most European funding bodies have also turned down panitumumab for second line treatment for colorectal cancer. However, the published study does not reflect the questions over the statistical analyses, concluding that panitumumab “significantly improved” progression-free survival and there was “a trend toward” improved overall survival.4 Subsequent hype and misleading reporting can put pressure on governmental bodies to reimburse a drug. Health technology assessors are in an invidious position—if they refuse to reimburse and a drug later turns out to be an important therapeutic advance, then patients have lost out because of the delay. If they reimburse and the drugs later turn out to be ineffective, have no clinically meaningful effects, or not to be cost effective then patients may have been unnecessarily subjected to toxic drugs and scarce healthcare resources have been wasted. The combination of this hype coupled with underlying concerns about the quality of trials can cause confusion. In England, pomalidomide, an immunomodulatory drug for refractory multiple myeloma wasn’t available, then was, then wasn’t, then was again. The drug was approved by the drug regulators both in Europe and the US despite questions over trial design (box 1). Correspondence between the FDA and Celgene, the trial sponsor, shows the US agency cautioning the company that its choice of comparator was unsuitable for regulatory decision making.1112 The FDA ultimately ignored its own advice and approved the drug regardless. The EMA approved it several years later. When NICE rejected the drug for use in the NHS because of the poor comparator, Celgene claimed that it was chosen only after consulting with the regulators. Pomalidomide subsequently went onto England’s Cancer Drugs Fund, an extra source of funding, only for budgetary constraints eventually leading to it being removed in 2015. It was one of 17 drugs for 23 different indications to be included in the cull.13 The delisting prompted both companies and charities to call for all interested parties to work together to find a better solution for patients, putting pressure on NICE. Noticeably, however, this did not necessarily include better evidence generation and better oversight from the EMA. “This has been a unilateral decision by NHS England. What is missing is a willingness for all stakeholders to take part in collaborative discussion and work together,” Wim Souverijns, general manager at Celgene UK and Ireland, said. “There is a role for companies to put pressure on stakeholders, including NICE and NHS England, to point out the implications of these changes.”13 This pressure may have worked for patient access and cost purposes. In 2016—after the company offered a confidential discount—NICE approved the drug because it then became cost effective. But that does not mean the evidence of efficacy is robust; other European countries continue to deem it not worth paying for.

Regulator fit for purpose? The inability of EMA to uphold its current policies on drug approval has implications for patients and budgets, and the scientific advice the agency gives is coming under increasing scrutiny (box 2). The example of bevacizumab (Avastin) is a cautionary tale about how well the EMA can monitor, evaluate, and learn about products even after they are on the market. Yet the agency wants to push ahead with a programme allowing quicker access to drugs with immature data.14 In June 2011, the FDA announced it had revoked bevacizumab’s indication for metastatic breast cancer because it “has not been shown to provide a benefit, in terms of delay in the growth of tumors, that would justify its serious and potentially life-threatening risks. Nor is there evidence that [it] will either help women with breast cancer live longer or improve their quality of life.”15 But the drug is still licensed in Europe for metastatic breast cancer; the EMA withdrew the licence for only some uses. It said this was because the available data on use in combination with paclitaxel have “convincingly shown [the drug] to prolong progression-free survival of breast cancer patients without a negative effect on the overall survival.”16 Again the EMA’s decision caused challenges for funders. Bevacizumab was rejected by NICE and was one of the most requested drugs under the Cancer Drugs Fund.17 It was another of the drugs delisted in 2015, leading to disappointment of patient groups. “People with incurable breast cancer can only watch from the sidelines as life-extending treatments are debated again and again and vital options disappear,” the charity Breast Cancer Care said in a statement.18 But patients will continue to have their hopes dashed if regulators approve drugs using designs that are not methodologically rigorous.

Footnotes Competing interests: I have read and understood BMJ policy on declaration of interests and have no relevant interests to declare.

Provenance and peer review: Commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.