In order to increase the likelihood that drug efficacy signals can be detected during clinical trials, pharmaceutical developers purposefully enroll subjects who are relatively homogenous. This procedural step, while vital for achieving robust statistical descriptions of a compound’s efficacy, necessarily leaves open the possibility that the test agent will have unexpected actions once it is used in a heterogeneous population of patients.

Often, serious and life-threatening side effects that were not exposed during preapproval screening programs become evident only after drug approval. A member of the U.S. Food and Drug Administration’s (FDA) Office of Drug Safety summed up the issue by stating, “The complete adverse event profile of a drug is not known at the time of approval because of the small sample size, short duration, and limited generalizability of pre-approval clinical trials.” Also, “since most trials exclude the elderly, children, pregnant women, patients with multiple diseases, and those on medications suspected of interaction with the study drug, the studies’ participants may not be representative of the real world where the drug is eventually used.”1

The gradual evolution of side-effect profiles across numerous drug classes only after they win FDA approval serves to underscore the preceding points (examples include severe cardiac complications from the weight management drug Meridia,2 a fatal muscle-wasting syndrome from the cholesterol management drug Baycol,3 and increased heart attack and stroke rates in patients taking Vioxx, prescribed for osteoarthritis and joint pain4). In short, careful postapproval monitoring is vital to the ongoing drug evaluation process.

Physicians and pharmacists routinely obtain safety information from drug label “inserts” that are often based predominately on preapproval clinical trial results. It is this reliance on incomplete safety data derived from limited clinical trial systems that can contribute to a significant gap in knowledge for practicing health care providers.1,5,6

The FDA’s Adverse Event Reporting System (FAERS) is a centralized computerized information database that is used for postmarketing drug safety surveillance. FAERS is currently growing by approximately 1,500,000 new cases per year.7 FDA professionals and pharmacovigilance experts routinely look to FAERS data as a guide to, and signal generator of, drug safety issues. Both groups employ a wide array of sophisticated data mining and signal detection techniques (for reviews, see Tatonetti et al., 2012, and Harpaz et al., 20128,9). The FDA uses such analyses to issue warnings, mandate label changes, and remove drugs from the U.S. market after an incidence, or severity, of their side effects is determined to significantly differ from what preapproval clinical trial results previously suggested.10 FAERS, and other similar spontaneous reporting systems maintained by governmental and international organizations, are a main resource for identifying postmarketing drug safety concerns.1,11-25

Detailed FAERS data, however, are neither readily accessible nor organized in a way that is useful to broad health care audiences. We believe that the data contained in FAERS represent valuable insight into real-world drug safety risks. Accordingly, we built the drug scoring system described in this article to provide a readily accessible evaluation of a drug’s potential safety risks. The system is based on serious adverse events (AEs) and patient outcomes listed in postmarketing case reports. It assesses drug safety by estimating the magnitude of downstream direct medical costs based on AE and outcome costing data taken from the Agency for Healthcare Research and Quality’s (AHRQ) Healthcare Cost and Utilization Project (HCUP).26,27

We observed that the minimum direct medical cost per patient exposure was approximately $0.02, and the maximum was approximately $10,000. These values appeared to be distributed exponentially. Accordingly, the raw cost data were transformed using the natural log to approximate a normal distribution. The minimum log-adjusted direct medical cost per patient was -4 and the maximum was 9.2. To scale those values to form a 1-100 scale, we used the following formula: (ln(x)+4)*7.5.

For each drug we selected, all reported primary suspect AEs from January 2010 through December 2014 were obtained. In cases with more than 1 eligible AE, only the AE with the largest individual cost was selected for each case. For example, it was sometimes observed that similar AEs were listed in a single case report (e.g., cerebral hemorrhage at $21,273 and ischaemic stroke at $14,858, or pulmonary embolism at $14, 878 and pulmonary infarction at $10,804), so we decided a “most costly” AE selection would better align with actual medical expenditures. In cases with no eligible AE, but with a listed outcome, we selected only the largest outcome direct medical cost per case. We divided the total direct medical costs derived from a drug’s case reports by the number of patients exposed over the same time period to obtain a direct medical cost per drug. Patient usage data for 2010, 2011, and 2012 were based on information derived from the Medical Expenditure Panel Survey (MEPS). 37 Given that MEPS is only available through 2012, we used sales-based figures provided by Evaluate Pharma for 2013 and 2014. These figures are estimates of the number of patients in the U.S. market receiving a drug in a given year. They are calculated based on disclosed U.S. sales divided by the revenues per patient per year (cost per patient, adjusted for patient compliance rate [%], and off-invoice discounts).

Given that the FAERS database is MedDRA coded, we used BioPortal, a repository of biomedical ontologies, and an ICD-9-CM mapping resource to assign ICD-9-CM codes to MedDRA Preferred Term (PT) AEs. 33 , 34 For case reports in which no eligible direct medical cost was assigned, we used AHRQ “outcome” figures. 35 We were able to assign costs to 1,508 serious MedDRA PT AEs. Of those PT AEs, 1,213 (80%) were mapped using BioPortal to ICD-9-CM codes with available cost data; 295 PT AEs (20%) that could not be assigned by the use of BioPortal were manually mapped using the ICD-9-CM coding manual with the following hierarchy 34 : (a) verbatim match (e.g., MedDRA PT “asphyxia” was mapped to ICD-9-CM code 799.01, asphyxia); (b) PTs matched to broader ICD-9-CM categories (e.g., MedDRA PT “nephrogenic systemic fibrosis” was mapped to ICD-9-CM code 710.8, other specified diffuse diseases of connective tissue); and (c) for terms that were mapped to multiple ICD-9-CM categories, we obtained a weighted average of the relevant direct medical cost data (e.g., MedDRA PT “cardio-respiratory arrest” was mapped to ICD-9-CM codes 427.5, cardiac arrest, and 799.1, respiratory arrest). Appendix B (available in online article) shows examples of costs and ICD-9-CM mapping.

HCUP is a compilation of patient data collected by the AHRQ, that is coded to the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). 26 , 27 We used HCUP to obtain national hospitalization and aggregate costs for specific diagnoses and procedures.

AE information was coded according to MedDRA version 17.1. 29 “Primary suspect” designations in FAERS case reports were quantified in an attempt to restrict the analysis to those drugs directly suspected of causing the AE. A discrepancy occasionally observed with FAERS case reports is that disease-related symptoms are sometimes listed in the “adverse event” field. In instances where such mistakes were easily identifiable, we excluded those “AEs” from analysis. For example, approximately 1% of the case reports for pramipexole dihydrochloride and approximately 1% of the case reports for donepezil hydrochloride listed Parkinson’s disease and dementia, respectively, as an AE. Such case reports are not included in our analyses.

Established Pharmacological Class (EPC) is a designation found in the FDA’s National Drug Code file that indicates an established pharmacologic class, as required by the FDA’s structured product labeling process. 32 All drugs were sorted into their corresponding EPCs. Score averages were calculated for each EPC.

Drug name text-mapping was accomplished as described by Hoffman et al. (2013). 28 Drug names were normalized to RxNorm reference codes using string searching and manual curation. 30 National Drug File Reference Terminology was used to provide ancillary information on class and mechanism of action. 31

Case reports that were missing or contained malformed key identification fields (Individual Safety Report number [ISR], patient number, drug sequence identification, or AE terms from the Medical Dictionary for Regulatory Activities [MedDRA] 29 ) were discarded. As long as the previously mentioned key identification fields were contained in a given case report, allowable missing fields included age, gender, weight, outcome, and condition. Cases were discarded if the drug name was found to be indeterminate or if the name was determined to not represent an FDA-approved drug (e.g., dietary supplements or foods). In instances where there was more than 1 Individual Case Safety Report (ICSR) for the same identification number in the same calendar year, the earliest reported case was selected ( Appendix A , available in online article).

To import and filter data from FAERS, common data preprocessing techniques were used to normalize and qualify textual data, such as removal of nonalphanumeric characters, white spaces, and line breaks. Filtering processes included the following: (a) a system for automated name matching that corrected for drug name misspellings and incorrect data within major fields (i.e., the inclusion of dosages or routes of administration as part of the drug name field); (b) aggregation of generic and non-U.S. brand name drugs under a single brand name; (c) separation of “primary suspect” and “all suspect” designations; and (d) identification of common AE and condition types. Automated data preprocessing and scrubbing workflow provided an initial assignment of “raw” FAERS drug names. The automated matching process utilized a combination of fuzzy string matching, string distance, and phonetic matching algorithms.

AE and patient outcome data were obtained for 706 FDA-approved drugs during the time period of January 2010 through December 2014 from an analytic system of FAERS case reports. 28 Nonserious and disease-related AEs were ignored, as well as nonserious outcomes.

To determine which ATC groups were associated with a high percentage of elevated scores, we plotted the percentage of scores that were 60 and above for each group with 10 or more individual drugs. Appendix D (available in online article) shows that antineoplastic drugs were outliers with approximately 80% of their individual scores at 60 and above. Blood and anti-infective drugs had the second and third highest percentage (~20%-30%) of scores that were 60 and above. In contrast, respiratory, genitourinary, and cardiovascular groups had the lowest percentage of scores that were 60 and above, respectively.

In Figure 2 , individual drug scores were mapped to their corresponding Anatomical Therapeutic Chemical (ATC) codes, another widely used drug classification system based on the site of drug action as well as pharmacologic, chemical, and therapeutic properties. 38 Only ATC groups with 10 or more individual drug members are included in this figure.

In an attempt to highlight EPCs that might pose a high level of risk to a large number of patients, Appendix C (available in online article) shows the distribution of weighted average scores for each EPC that comprised 3 or more individual compounds and had 3,000 or more case reports over the time period studied. The tumor necrosis factor blocker EPC was an outlier because of a high number of case reports combined with a high score average. Other EPCs with the combination of a weighted average above 60 and over 20,000 costed cases were vascular endothelial growth factor-directed antibody, recombinant human interferon beta, kinase inhibitor, and endothelin receptor antagonist, suggesting that they represent elevated risks across large populations of patients. EPCs with lower risk and lower case counts were penicillin-class antibacterial, angiotensin-converting enzyme inhibitor, beta-adrenergic blocker, and thiazide diuretic.

EPC is a designation found in the FDA’s National Drug Code file that indicates an established pharmacologic class, as required by FDA’s structured product labeling requirements. 32 All drugs were sorted into their corresponding EPC classes. Endothelin receptor antagonists and kinase inhibitors were the 2 EPCs with the highest weighted averages ( Table 1 ). Other EPCs with median scores of 70 and above included hepatitis C virus NS3/4A protease inhibitor, recombinant human interferon beta, vascular endothelial growth factor-directed antibody, tumor necrosis factor blocker, alkylating drug, and microtubule inhibitor. Table 2 shows 15 of the highest scoring EPCs where each class had 3,000 or more cases reports and 3 or more individual drug members.

Factors contributing to high scores were elevated associations with significant AEs. For example, the 2 highest scoring drugs were thalidomide analogs, with pomalidomide having 241 case reports of pneumonia ($2,099,833 total; $8,713 each); 90 reports of pancytopenia ($1,032,840 total; $11,476 each); and 82 reports of neutropenia ($990,806 total; $12,083 each). Lenalidomide had 1,673 case reports of pneumonia ($14,576,849 total; $8,713 each); 889 reports of pancytopenia ($10,202,164 total; $11,476 each); and 811 reports of neutropenia ($9,799,313 total; $12,083 each). For the 2 highest scoring endothelin receptor antagonists, bosentan had 477 case reports of pneumonia ($4,156,101 total; $8,713 each); 245 reports of congestive cardiac failure ($2,492,385 total; $10,173 each); and 231 reports of respiratory failure ($4,296,369 total; $18,599 each), while ambrisentan had 1,220 case reports of pneumonia ($10,629,860 total; $8,713 each); 464 reports of congestive cardiac failure ($4,720,272 total; $10,173 each); and 256 reports of respiratory failure ($4,761,344 total; $18,599 each).

The total number of drugs included in this analysis was 706. The minimum score was 8.29 (cost of $0.02) and the maximum score was 99.25 ($10,220). The median, mean, and standard deviation were 40.58 ($4.10), 44.45 ($6.87), and 18.29 ($0.21), respectively. There were 79 drugs with scores of 70 and above ($207.13), while 131 drugs had scores of 60 and above ($54.60). Figure 1 shows the distribution of scores across all 706 drugs, and Table 1 shows the top 50 highest scores, corresponding EPC, and the number of case reports analyzed. While the top 50 drugs comprised various drug classes, the following classes had more than 1 drug listed in the top 50: kinase inhibitor (14 individual drugs), endothelin receptor antagonist (3), nucleoside metabolic inhibitor (2), proteasome inhibitor (2), thalidomide analog (2), and vascular endothelial growth factor-directed antibody (2).

Discussion Section: Choose Top of page Abstract Methods Results Discussion << Conclusions REFERENCES CITING ARTICLES

Preapproval clinical trial participants are subjected to rigorous inclusion and exclusion criteria in order to increase the likelihood that drug efficacy signals can be detected during clinical testing. Such selection processes are vital for determining a compound’s efficacy and are also usually necessary for financial and logistical reasons. The downside of such methods, especially with regard to determining the safety profile of a drug, is that enrolled subjects are relatively homogenous. The preapproval clinical trial process, therefore, often fails to uncover many of the side effects that occur once the drug is introduced to real-world, heterogeneous patient populations. Accordingly, AEs from drugs approved by the FDA are a major public safety concern. In fact, approximately 1,500,000 AE reports are currently submitted to FAERS, the FDA’s repository of postmarketing AEs each year, and this figure is widely believed to be an underestimation of the actual amount of AEs triggered by approved drugs.7

When performing comparative effectiveness research, health care decision makers routinely obtain safety information from drug label inserts, which are dossiers prepared by pharmaceutical companies, and preapproval clinical trial results. Such data are limited in scope by publication bias, commercial interests, disclosure delays, and the homogeneity of test subjects.

Drug label inserts are largely based on clinical trial data, but such data are associated with serious publication bias.39-45 For example, in a large study of over 100 clinical trials, 65% of harms outcomes were incompletely reported.46

Pharmacy and therapeutics (P&T) committees are subjected to pharmaceutical company bias, since they rely heavily on dossiers prepared by drug manufactures. Tellingly, only 43% of respondents in a recent Ernst & Young survey agreed with the statement that “pharmaceutical companies have data that are credible for measuring and improving outcomes.”47 Additionally, while “curbing rising drug costs” was one of the biggest concerns noted in the same survey, “curbing rising medical (non-drug) costs” was the second biggest payer concern. Finally, boosting drug adherence was noted as a critical component of lowering health care costs. The lack of real-world AE data hampers P&T decisions aimed at addressing these issues.

Accordingly, there is an obvious gap in drug safety information flow regarding real-world medication use. One way to address the deficiencies associated with the clinical trial system and currently available drug safety information would be to enhance the use of postapproval AE data.

Numerous studies have documented the utility of FAERS for generating safety signals.13,14,17-21,23,24,48-50 Accordingly, the benefit of using FAERS data to fill information gaps left by preapproval safety testing appears clear. However, meaningful analysis of FAERS largely depends on expensive, complex, and proprietary data mining and signaling tools. These limitations block routine FAERS data usage by most health care decision makers.

Therefore, in order to make FAERS data accessible and useful for health care decision makers, we developed an analytic to estimate direct medical costs associated with AEs and patient outcomes linked to 706 FDA-approved drugs. Data were derived from postmarketing reports submitted to the FDA rather than from preapproval clinical trials. The system combines AHRQ costs and FAERS data to estimate downstream direct medical costs. The 2 highest individual scores were attributed to thalidomide analogues, while 14 of the top 50 highest scores were attributable to kinase inhibitors. In general, EPC drug classes with the highest average scores were kinase inhibitor, endothelin receptor antagonist, hepatitis C virus NS3/4A pro-tease inhibitor, recombinant human interferon beta, vascular endothelial growth factor-directed antibody, and tumor necrosis factor blocker. ATC drug class results of note included the findings that antineoplastic drugs were outliers with approximately 80% of their individual scores at 60 and above, as well as blood and anti-infective drugs with approximately 20%-30% of their scores at 60 and above. While some drug classes had a narrow range of scores, certain within-drug class analyses demonstrated broad score ranges for similar medications.

We believe this system provides an accessible reference point regarding real-world differences in safety observed during a drug’s postmarketing phase. Because the scoring system is based on direct medical costs, it may be used to improve patient safety by identifying medications that cause undue burdens on patients and health care providers. These scores can be used for identification of drugs that are driving increased costs, as well as assessments and comparisons within a given mechanism of action, across nonrelated drugs, within a drug class, or within a treatment indication.

Within managed care, the assessment of formulary or preferred status from an individual organizations safety data could be improved by including estimated direct medical costs from AEs and patient outcomes from millions of case reports contained in FAERS. The scores could be used to help health care decision makers drive safer prescribing behavior by optimizing the construction of drug formularies, budget impact models, and by adding an AE component to population health initiatives. Within-drug class score differences provide head-to-head comparisons that can be utilized for tier placements as well as prior authorization and step-therapy decisions.