The reversal of established medical practice is common and occurs across all classes of medical practice. This investigation sheds light on low-value practices and patterns of medical research.

We reviewed 2044 original articles, 1344 of which concerned a medical practice. Of these, 981 articles (73.0%) examined a new medical practice, whereas 363 (27.0%) tested an established practice. A total of 947 studies (70.5%) had positive findings, whereas 397 (29.5%) reached a negative conclusion. A total of 756 articles addressing a medical practice constituted replacement, 165 were back to the drawing board, 146 were medical reversals, 138 were reaffirmations, and 139 were inconclusive. Of the 363 articles testing standard of care, 146 (40.2%) reversed that practice, whereas 138 (38.0%) reaffirmed it.

We reviewed all original articles published in 10 years (2001-2010) in one high-impact journal. Articles were classified on the basis of whether they addressed a medical practice, whether they tested a new or existing therapy, and whether results were positive or negative. Articles were then classified as 1 of 4 types: replacement, when a new practice surpasses standard of care; back to the drawing board, when a new practice is no better than current practice; reaffirmation, when an existing practice is found to be better than a lesser standard; and reversal, when an existing practice is found to be no better than a lesser therapy. This study was conducted from August 1, 2011, through October 31, 2012.

We expect that new medical practices gain popularity over older standards of care on the basis of robust evidence indicating clinical superiority or noninferiority with alternative benefits (eg, easier administration and fewer adverse effects). The history of medicine, however, reveals numerous exceptions to this rule. Stenting for stable coronary artery disease was a multibillion dollar a year industry when it was found to be no better than medical management for most patients with stable coronary artery disease.Hormone therapy for postmenopausal women intended to improve cardiovascular outcomes was found to be worse than no intervention,and the routine use of the pulmonary artery catheter in patients in shock was found to be inferior to less invasive management strategies.Previously, we have called this phenomenon (when a medical practice is found to be inferior to some lesser or prior standard of care) a medical reversal.Medical reversals occur when new studies—better powered, controlled, or designed than their predecessors—contradict current practice.In a prior investigation of 1 year of publications in a high-impact journal, we found that of 35 studies testing standard of care, 16 (46%) constituted medical reversals.Another review of 45 highly cited studies that claimed some therapeutic benefit found that 7 (16%) were contradicted by subsequent research.

Writing Group for the Women's Health Initiative Investigators Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial.

Identifying medical practices that do not work is necessary. The continued use of such practices wastes resources, jeopardizes patient health, and undermines trust in medicine. Interest in this topic has grown in recent years. The American Board of Internal Medicine launched the Choosing Wisely campaign,a call on professional societies to identify the top 5 diagnostic or therapeutic practices in their field that should not be offered.In England, the National Institute for Health and Clinical Excellence has tried to “disinvest” from low-value practices, identifying more than 800 such practices in the past decade.Other researchers have found that scanning a range of existing health care databases can easily generate more than 150 low-value practices.Medical journals have specifically focused on instances in which more health care is not necessarily better. The Archives of Internal Medicine created a new feature series in 2010 entitled “Less is More.”

Less is more: how less health care can result in better health.

Given ongoing and vigorous efforts to identify medical practices that offer little benefit and minimal empirical studies documenting the rate at which current practices are contradicted, we performed a review of 10 years of original publications in one high-impact journal.

Data are summarized using counts and percentages. A linear regression was performed to determine the relationship between percentage of reversals and time, and the Pearson χ 2 test was used when appropriate. Analyses were conducted using Stata statistical software, version 12 (StataCorp LP).

Two reviewers (C.T., A.V., M.C., J.R., S.Q., S.J.C., D.B., V.G., or S.S.) and V.P. read articles addressing a medical practice in full. On the basis of the abstract, introduction, and discussion, articles were classified as to whether the practice in question was new or existing. Methods were classified as one of the following: randomized controlled trial, prospective controlled (but nonrandomized) intervention study, observational study (prospective or retrospective), case-control study, or other methods. End points for articles were classified into those that reached positive conclusions and those that found negative or no difference in end points. Lastly, articles were given 1 of 4 designations. Replacement was defined as a new practice surpassing an older standard of care. Back to the drawing board was defined as a new practice failing to surpass an older standard. Reversal was designated when a current medical practice was found to be inferior to a lesser or prior standard. Reaffirmation was defined as an existing medical practice being found to be superior to a lesser or prior standard. Finally, articles in which no firm conclusion could be reached were termed inconclusive. The designation of an article was also performed in duplicate. When there were differences in opinion between the 2 reviewers, adjudication first involved discussion between the 2 readers to see whether agreement could be reached. If disagreement persisted, a third reviewer (A.C.) adjudicated the discrepancy. Less than 3% of articles required discussion, and less than 1% required adjudication. A table detailing each medical reversal was constructed ( Supplemental Appendix ; available online at http://www.mayoclinicproceedings.org ), and the third reviewer (A.C.) reviewed all reversals.

On the basis of published abstracts, articles were classified as to whether they addressed a clinical practice. Articles addressing a medical practice were defined as any investigation that assesses a screening, stratifying, or diagnostic test, a medication, a procedure or surgery, or any change in health care provision systems. Many research articles concern the novel molecular basis of disease or novel insights in pathophysiology. These articles were excluded. When practice information could not be ascertained by abstract alone, full articles were read.

We used methods similar to our prior survey of 1 year of publications in a high-impact journal.We reviewed all articles under the heading “Original Articles” in the New England Journal of Medicine from 2001 to 2010. These years were the last complete 10 years when we began our investigation. Our choice of journal was made on the basis of the 5-year Hirsch index for medical journals.Two reviewers (C.T., A.V., M.C., J.R., S.Q., S.J.C., D.B., V.G., or S.S.) and V.P. independently extracted information for each calendar year. This study was conducted from August 1, 2011, through October 31, 2012.

lists the 10 selected reversals in the decade and how each article contradicted current standard of care. The Supplemental Appendix details all 146 reversals. Figure 2 shows the percentage of articles that tested standard of care and, of those, the percentage of reversals and reaffirmations. The percentage of reversals among articles that tested standard of care were constant during the decade (P=.51).

American College of Cardiology/American Heart Association Task Force on Practice Guidelines; American College of Cardiology/American Heart Association/Society for Cardiovascular Angiography and Interventions Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention

American College of Cardiology/American Heart Association Task Force on Practice Guidelines; American College of Cardiology/American Heart Association/Society for Cardiovascular Angiography and Interventions Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention ACC/AHA/SCAI 2005 Guideline Update for Percutaneous Coronary Intervention–summary article: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (ACC/AHA/SCAI Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention).

American College of Cardiology; American Heart Association Task Force on practice guidelines (Committee on the Management of Patients With Chronic Stable Angina) ACC/AHA 2002 guideline update for the management of patients with chronic stable angina–summary article: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines (Committee on the Management of Patients With Chronic Stable Angina).

The use of hypothermia as a method of neuroprotection during neurosurgical procedures and after traumatic brain injury: a survey of clinical practice in Great Britain and Ireland.

High-dose chemotherapy and stem-cell rescue in the treatment of high-risk breast cancer: prognostic indicators of progression-free and overall survival.

American College of Cardiology/American Heart Association Task Force on Practice Guidelines; American College of Cardiology/American Heart Association/Society for Cardiovascular Angiography and Interventions Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention

American College of Cardiology/American Heart Association Task Force on Practice Guidelines; American College of Cardiology/American Heart Association/Society for Cardiovascular Angiography and Interventions Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention ACC/AHA/SCAI 2005 Guideline Update for Percutaneous Coronary Intervention–summary article: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (ACC/AHA/SCAI Writing Committee to Update the 2001 Guidelines for Percutaneous Coronary Intervention).

American College of Cardiology; American Heart Association Task Force on practice guidelines (Committee on the Management of Patients With Chronic Stable Angina) ACC/AHA 2002 guideline update for the management of patients with chronic stable angina–summary article: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines (Committee on the Management of Patients With Chronic Stable Angina).

The use of hypothermia as a method of neuroprotection during neurosurgical procedures and after traumatic brain injury: a survey of clinical practice in Great Britain and Ireland.

High-dose chemotherapy and stem-cell rescue in the treatment of high-risk breast cancer: prognostic indicators of progression-free and overall survival.

Eight of the reversals we identified overlapped with an Australian study of 156 low-value practices Supplemental Figure ; available online at http://www.mayoclinicproceedings.org ). These reversals include arthroscopic surgery for knee osteoarthritis,vertebroplasty for osteoporotic fractures,endovascular repair of infrarenal abdominal aortic aneurysms,stenting in patients with stable coronary artery disease,amnioinfusion for women with meconium staining,C-reactive protein testing,screening men with the prostate specific antigen test,and routine revascularization or stress testing before surgery.Thus, we provide at least 138 unique low-value practices.

Three articles further contradicted routine hormone therapy in postmenopausal women.Two articles contradicted routine use of the pulmonary artery catheter,and 2 articles found worse outcomes with recommended glycemic targets (as opposed to more permissive standards) for patients with diabetes.The benefit of stenting in patients with stable coronary artery disease was undermined by the Occluded Artery Trial,Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluationtrial, and a follow-up quality-of-life study from the Occluded Artery Trial.Two studies suggested that although ezetimibe improves low-density lipoprotein values, it does not improve carotid artery intima media thickness.Arthroscopic surgery of the knee for osteoarthritis was called into question by 2 studies 5 years apart,whereas vertebroplasty for osteoporotic fracture was contradicted by 2 paired articles.Adjusting for the fact that several reversals concerned the same practice, 128 medical practices were contradicted during these 10 years.

Several of the reversal articles concerned the same topic. Four articles called into question the drug aprotinin,which was widely used in cardiac surgery but found to increase mortality. Three articles addressed use of a primary rhythm control strategy for patients with atrial fibrillation.Three articles in a single issue found increased risks of cardiovascular events from using the cyclooxygenase 2 inhibitors, including rofecoxib.Three articles provided extended follow-up for a trial of children randomly assigned to early myringotomy with the insertion of tympanostomy tubes or a delayed procedure. Although the procedure was the most common operation performed on children beyond the newborn periodand bolstered by expert guidelines,no difference was found in an early vs delayed strategy on outcomes at 3,6,or 9 to 11 years of age.

Effect of early or delayed insertion of tympanostomy tubes for persistent otitis media on developmental outcomes at the age of three years.

Effect of early or delayed insertion of tympanostomy tubes for persistent otitis media on developmental outcomes at the age of three years.

Articles that tested new practices were more likely to find them beneficial than articles that tested existing ones (77.1% vs 38.0%; P<.001). Conversely, articles that tested existing standards were more likely to find those practices ineffective than articles testing new practices (40.2% vs 17.0%; P<.001).

Of the 363 articles that tested an existing medical practice, 146 (40.2%) found it ineffective compared with a previous standard or its omission (reversals), whereas 138 (38.0%) upheld the practice, and 79 (27.3%) were inconclusive. Table 1 and Figure 2 provide, for articles testing existing standard of care, a breakdown of reversal, reaffirmation, and inconclusive articles by year. Of the 146 reversal articles, most were randomized controlled trials (111 [76.0%]); 13 (8.9%) were prospective, nonrandomized studies; 20 (13.7%) were retrospective studies; 1 was a case-control study; and 1 used an alternative study design.

Concerning the study results, 947 (70.5%) reached positive conclusions, whereas 397 (29.5%) reached negative conclusions or found no difference between comparators. As such, 756 articles (56.3%) found a new practice surpassing current standard of care (replacement), 165 (12.3%) found a new practice failing to improve on the current practice (back to the drawing board), 146 (10.9%) were reversals, and 138 (10.3%) upheld standard of care over a lesser or prior standard (reaffirmation). A total of 139 (10.3%) were deemed inconclusive. Figure 1 shows a breakdown of articles. The single most common study type was a randomized trial examining a new practice and finding benefit for that practice; 530 (39.4%) of all 1345 articles were classified as such.

From 2001 through 2010, 2044 original articles appeared in one high-impact journal. Most articles (1344 [65.8%]) addressed a medical practice. A total of 981 studies (73.0%) examined a new medical practice, whereas 363 (27.0%) addressed an existing practice. During these 10 years, there were 911 (67.7%) randomized controlled trials, 220 (16.4%) prospective controlled but nonrandomized studies, 117 (8.7%) observational studies, 43 (3.2%) case-control studies, and 53 (3.9%) studies using other methods.

Discussion

Our review of 10 years of publications in a high-impact journal involved examining 2044 articles in duplicate to identify 146 medical reversals. Reversals included medications, procedures, diagnostic tests, screening tests, and even monitoring and treatment guiding devices. We were unable to identify any class of medical practice that did not have some reversal of standard of care ( Supplemental Appendix ).

74 Lennmarken C.

Bildfors K.

Enlund G.

Samuelsson P.

Sandin R. Victims of awareness. 75 Myles P.S.

Leslie K.

McNeil J.

Forbes A.

Chan M.T.V. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. 76 Puri G.D.

Murthy S.S. Bispectral index monitoring in patients undergoing cardiac surgery under cardiopulmonary bypass. 77 Lang J. Awakening. The Atlantic. January/February 2013. 78 Avidan M.S.

Zhang L.

Burnside B.A.

et al. Anesthesia awareness and the bispectral index. 4 Prasad V.

Gall V.

Cifu A. The frequency of medical reversal. 79 Hall J.B. Use of the pulmonary artery catheter in critically ill patients: was invention the mother of necessity?. , 80 Tatsioni A.

Bonitsis N.G.

Ioannidis J.P. Persistence of contradicted claims in the literature. The bispectral index monitor (BIS) illustrates many of the principles of medical reversal. Although rare, anesthesia awareness (or intraoperative awareness) is debilitating and is associated with posttraumatic stress disorder and anxiety.The BIS monitor was developed to ensure that patients were receiving adequate anesthesia by using a single electroencephalographic lead to calculate a dimensionless measure of consciousness. In theory, anesthesia could be titrated to the BIS reading. In 1997, the US Food and Drug Administration approved the device. Only 2 trials existed before the reversal study. One, an industry-sponsored trial, did not use a standardized protocol for the comparator arm and found the device reduced awareness.The other was underpowered to make any conclusions.Nevertheless, the monitor's use increased. By July 2007, half of all operating rooms in the United States had a BIS monitor.Then in 2008, a large, randomized trial comparing the BIS monitor with a standardized sedation monitoring strategy found no benefit for the device on anesthesia awareness.Many reversals have similar narratives.Although there is a weak evidence base for some practice, it gains acceptance largely through vocal support from prominent advocates and faith that the mechanism of action is sound. Later, future trials undermine the therapy, but removing the contradicted practice often proves challenging.Although the BIS monitor was designed to prevent a rare event (anesthesia awareness), many reversals concern common end points, such as mortality.

81 What conclusions has Clinical Evidence drawn about what works, what doesn't based on randomised controlled trial evidence? http://clinicalevidence.bmj.com/x/set/static/cms/efficacy-categorisations.html. Accessed June 30, 2011. Recently, a project of BMJ, entitled Clinical Evidence,completed a review of 3000 medical practices. The project found that slightly more than a third of medical practices are effective or likely to be effective; 15% are harmful, unlikely to be beneficial, or a trade-off between benefits and harms; and 50% are of unknown effectiveness. Our investigation complements these data and suggests that a high percentage of all practices may ultimately be found to have no net benefits.

4 Prasad V.

Gall V.

Cifu A. The frequency of medical reversal. , 5 Prasad V.

Cifu A.

Ioannidis J.P.A. Reversals of established medical practices: evidence to abandon ship. , 6 Prasad V.

Cifu A. Medical reversal: why we must raise the bar before adopting new technologies. , 82 Prasad V.

Cifu A. A medical burden of proof: towards a new ethic. 4 Prasad V.

Gall V.

Cifu A. The frequency of medical reversal. 5 Prasad V.

Cifu A.

Ioannidis J.P.A. Reversals of established medical practices: evidence to abandon ship. 6 Prasad V.

Cifu A. Medical reversal: why we must raise the bar before adopting new technologies. , 83 Prasad V.

Rho J.

Cifu A. The diagnosis and treatment of pulmonary embolism: a metaphor for medicine in the evidence-based medicine era. , 84 Prasad V.

Rho J.

Cifu A. The inferior vena cava filter: how could a medical device be so well accepted without any evidence of efficacy?. 82 Prasad V.

Cifu A. A medical burden of proof: towards a new ethic. 85 Kozauer N.

Katz R. Regulatory innovation and drug development for early-stage Alzheimer's disease. To our knowledge, this is the largest and most comprehensive study of medical reversal. Previously, we have considered the causes and consequences of reversal.When medical practices are instituted in error, most often on the basis of premature, inadequate, biased, and conflicted evidence,the costs to society and the medical system are immense.As such, we favor policies that minimize reversal. Nearly all such measures involve raising the bar for the approval of new therapiesand asking for evidence before the widespread adoption of novel techniques. In all but the rarest cases,large, robust, pragmatic randomized trials measuring hard end points (with sham controls for studies of subjective end points) should be required before approval or acceptance. Our position is in contrast to efforts to lower standards for device and drug approval,which further erodes the value of the regulatory process.

86 Confavreux C.

Suissa S.

Saddier P.

Bourdès V.

Vukusic S. Vaccines in Multiple Sclerosis Study Group

Vaccinations and the risk of relapse in multiple sclerosis. , 87 Ascherio A.

Zhang S.M.

Hernán M.A.

et al. Hepatitis B vaccination and the risk of multiple sclerosis. 88 Fine L.G.

Barnett E.V.

Danovitch G.M.

et al. Systemic lupus erythematosus in pregnancy. 89 Sánchez-Guerrero J.

Uribe A.G.

Jiménez-Santana L.

et al. A trial of contraceptive methods in women with systemic lupus erythematosus. , 90 Petri M.

Kim M.Y.

Kalunian K.C.

et al. OC-SELENA Trial

Combined oral contraceptives in women with systemic lupus erythematosus. 91 Goetzl L.M. ACOG Committee on Practice Bulletins-Obstetrics

ACOG Practice Bulletin. Clinical Management Guidelines for Obstetrician-Gynecologists Number 36, July 2002. Obstetric analgesia and anesthesia. 92 Wong C.A.

Scavone B.M.

Peaceman A.M.

et al. The risk of cesarean delivery with neuraxial analgesia given early versus late in labor. 93 Koo H.L.

Koo D.C.

Musher D.M.

DuPont H.L. Antimotility agents for the treatment of Clostridium difficile diarrhea and colitis. One surprising type of reversal we observed was potentially beneficial therapies being withheld because of unfounded concerns about their potential to cause harm. Long-standing concerns that vaccinations precipitate flare of multiple sclerosis led many physicians to omit this intervention, but the concerns were largely undermined by the results of 2 studies in 2001.Concerns that oral contraceptives increase lupus flares created reluctance to prescribe this class of medications to women. This practice may contribute to a higher rate of elective abortions among patients with lupus.In 2005, 2 trials reported that oral contraceptives do not increase lupus flares.Although the American College of Obstetrics recommended that epidural anesthesia be delayed until cervical dilation has reached 4 cm—out of concern that earlier administration increases rates of cesarean section—randomized trials reported that this fear was unfounded.Warnings that turned out to be wrong represent a unique form of reversal and raise questions about other dubious restrictions taken at face value, for instance, that patients with Clostridium difficile infection should not be treated with antimotility agents for fear of increasing rates of toxic megacolon.Discerning readers may yet identify other novel patterns of contradiction.

5 Prasad V.

Cifu A.

Ioannidis J.P.A. Reversals of established medical practices: evidence to abandon ship. The current study has several limitations. Our choice of journal was made on the basis of impact factor rankings; thus, we are unsure whether our results apply to all journals. As in any study of published research findings, one may wonder whether there exists a publication bias favoring certain studies, in this case, those that contradict standard of care. However, the testing of standard of care is rarely doneand accordingly is in itself noteworthy. It seems unlikely that there exists a selection filter against reaffirmation articles.

4 Prasad V.

Gall V.

Cifu A. The frequency of medical reversal. Our classification scheme was based on prior work,but others may have alternative preferences for grouping medical articles. Whether a medical practice was considered new or existing was decided on the basis of the article's abstract, introduction, and discussion. We did not perform an independent search to verify that existing practices were indeed in use and new practices were not. As such, we may have made errors both of inclusion and exclusion. Some authors may have chosen to downplay a therapy's real-world use, whereas others may have chosen to overemphasize it. An independent evaluation of practice patterns would have strengthened our investigation but would have been overly time-consuming because it would have required investigation of hundreds of topics, many of which are common medications that lack unique coding for their varying indications.

94 Ioannidis J.P.A. Why most published research findings are false. The reversals we have identified by no means represent the final word for any of these practices. Simply because newer, larger, better controlled or designed studies contradict standard of care does not necessarily mean that older practices are wrong and new ones are right. On average, however, better designed, controlled, and powered studies reach more valid conclusions.Nevertheless, the reversals we have identified at the very least call these practices into question. Some practices ought to be abandoned, whereas others warrant retesting in more powerful investigations. One of the greatest virtues of medical research is our continual quest to reassess it.