The aim of our review was to examine the usefulness and relevance of ADR information—for antidepressants and anticonvulsants—contained in product information for prescribers and patients. We compared the content between the USA and Europe for drugs marketed in both regions by the same market authorisation holder, to assess consistency. While some variability would be anticipated due to differences in regulatory cycles between regions and construction of the documents by different company teams, we would expect that the ADR profile between paired documents would be similar, as it should be based on centralised safety data.

Previous research has demonstrated inconsistencies in information included in the product documents. Studies have reported discrepancies across countries for drugs marketed by the same pharmaceutical company regarding the number of ADRs, and differing risk estimates and number of patients in the safety sample. 7–9 Other studies have found inconsistencies for the updating of contraindication labelling, pregnancy and lactation labelling, and advice on overdose in SmPCs compared to clinical management. 10–12

Comprehensive information on drug harm in clinical trials is now routinely collected as a result of improved regulation since the early 1990s. Extensive research has demonstrated that the collection and reporting of harm data in peer-reviewed published clinical trials are both still substandard and, as a consequence, the data are not adequate to inform risk-benefit decisions nor sufficient to contribute to cost-effectiveness analysis. 1–3 The main reasons for this are differing approaches in data collection, inadequate reporting of methodological details and selective harm outcome reporting. Poor quality harm data found in our systematic review of randomised trials for neuropathic pain led us to investigate other potential reliable sources of harm information. 4 The regulatory application for marketing authorisation requires drug manufacturers to list all known drug harm in a ‘product information document’ that is primarily written for use by healthcare workers. 5 , 6 The aim of this regularly updated document is to present a comprehensive picture, combining data from all available sources, of the harm profile gathered throughout the development, testing and use of the drug. The document should include any known harm discovered by manufacturers, who are the marketing authorisation holders, as well as external parties. Consequently, this publically available document is uniquely placed to be an invaluable source of information for patients, healthcare workers, researchers and regulators. This document is called the ‘summary of product characteristics' (SmPC), and is approved by the European Medicines Agency (EMA); and the US product insert (USPI) is approved by the US Food and Drug Administration (FDA). 5 , 6

Drug harm profiles are established from various sources. Spontaneous reports of suspected adverse drug reactions (ADRs) rely on healthcare workers or patients recognising an association between a drug and an adverse event (AE). Observational studies and electronic healthcare records can be useful for identifying rare ADRs due to large sample sizes, but they are subject to a number of biases and often have no robust comparator.

Prescribing choice between several available drug therapies is based on clinical efficacy with consideration given to the ‘perceived risk’ of the drug. The perceived risk plays an increasingly important role in treatment decisions when there is little difference in efficacy or when comparative effectiveness among competing treatments is unknown. Robust information on the drug harm profile is therefore vital to facilitate the informed risk-benefit decisions made by patients and prescribers.

As this is a descriptive study, appropriate measures of central tendency (mean, median) and dispersion (SD, range) were calculated for quantitative data, frequency and percentages for categorical variables. No significance tests were performed. The study protocol is available on request from the corresponding author.

Data extracted included: all ADRs; risk estimates; study design; sample size; the criterion used to identify ADRs from all AEs recorded in a clinical trial; dictionary used for coding ADRs, whether information on recurrent ADRs was reported; information on onset and duration of ADRs. Document level data were extracted by three reviewers (KL, VRC and OS) in duplicate, using a customised and piloted spreadsheet. Any differences in interpretation were resolved through discussion. Information on individual ADR terms was extracted electronically into a spreadsheet by a single reviewer (KL) and then checked by a second reviewer (VRC). Duplicate ADRs within a document were removed and the greater risk estimate was kept. ADRs due to rapid withdrawal or interactions with other medications were excluded from the review.

Regulatory guidelines from the FDA and EMA request that the manufacturers should report all ADRs, which are AEs where there is a reasonable possibility the drug has played a causal role, however, documents sometimes included the term ‘adverse event’ for which the accepted definition implies that the drug may not be causal. 5 , 6 As it was often not clear whether the events were truly ADRs or AEs, a pragmatic decision to extract all harm events reported was made—hereafter referred to as ADRs. Justification for this decision was based on the guidance by the EMA, which specifies that ‘adverse events, without at least a suspected causal relationship should not be reported’ and the FDA guidance, which stated (reporting of) ‘an adverse reaction not plausibly related to drug therapy should be avoided’. A summary of the reporting guidelines for the FDA and European Commission can be seen in online supplementary table A.

Documents were downloaded from September 2013–January 2015 from the US FDA and EMA, respectively. 13–15 The paired documents were chosen to ensure inclusion of the same dose and formulation, and downloaded at the same time.

Medical terminology dictionaries are used to code AE terms with an aim to standardise reports. For only one drug was the dictionary used reported in both documents, and it was different in each (see online supplementary table D). Six US and three European documents specified the dictionary used.

In clinical trials, many AEs will be recorded for each treatment and not all of these will be causally linked to the drug under investigation. Ideally, a signal detection method that is statistically underpinned should be used to flag signals of ADRs from all AEs reported in a clinical trial. None of the documents used a statistical method to flag a signal but nine US and three European documents reported the rule-based approach used. It is important to know what method has been applied as this will dictate the number and type of ADRs that are flagged. The full list of differing rule-based approaches used are reported in table 4 . It can be seen that rule-based methods were not the same in three paired documents and that seven US documents used multiple criteria that were used within the same document ( table 4 ).

The anticipated duration, severity and likelihood of recurrence of an ADR is important to patients and prescribers. Only one US document contained any information about duration of an ADR, four US documents presented some information on severity but the information was limited to only a few ADRs. None of the documents presented data regarding recurrence or reversibility ( table 2 ).

The causal mechanism of an ADR is often time dependent and the absolute risk will depend on the duration of exposure, therefore, information on the observation period is necessary for interpretation. Knowledge of the observation period also reveals how much is known about long-term use of the drug. Eight US and one European document contained information on the duration of the observation period for a couple of ADRs (see online supplementary table C) but only three documents from the USA reported this information for the majority of the ADRs arising from clinical trials ( table 2 ).

It is not possible to accurately estimate the risk of ADRs identified through spontaneous reports as the number of people who prescribed the drug is unknown. Crude estimates can sometimes be sought from sales or electronic health record data and can be used to calculate an approximate risk. 18 The European Commission guidance recommends that this estimate be based on ‘adequately designed study’ data, and with zero events observed, suggests estimating the upper 95% CI limit using 3/X, where X represents the total sample size summed up across all relevant clinical trials and studies (see online supplementary table A). It was unclear for all documents how risks of ADRs originating from spontaneous reports were derived. Nine of 12 US and 5 of 12 European documents reported risks for 80% or more of all ADRs originating from clinical trials ( table 2 ).

Knowing the source of the evidence of the ADR report (clinical trial, observational study, spontaneous report) provides important contextual information as both the study design and data collection method will directly impact on the estimated risk of the ADR. 16 , 17 The source of the ADR report was specified more frequently in the US documents than in the European documents. Of the 24 documents ten US and five European documents reported the source of the ADR report for 80% or more of all ADRs listed in the document, and the majority were reported as originating from clinical trial data ( table 2 and online supplementary table C).

On average, 77 more ADRs were reported in the US documents than in the European documents. The median numbers of ADRs were 114 (IQR 93, 150) and 201 (IQR 114, 262) in the European and US documents, respectively ( table 2 ). The majority of the ADRs terms differed between documents, on average only 29% of the total number of ADRs were reported in both documents ( table 3 ).

Nineteen anticonvulsant and antidepressant brand drugs were included as contenders for this review. Twelve of these were found to be concurrently marketed in Europe and the USA, with SmPC and USPI documentation available. The characteristics of the documents for the included drugs are listed in table 1 and all drugs identified from our previous review are listed in online supplementary table B.

Discussion

One of the aims of the FDA and EMA guidance for developing product information documents is to promote the consistency of harm reporting across drugs. It would be expected that the harm profiles of a drug marketed by the same company should be similar in the US and European document with the only differences due to variability in regulatory recommendations. This study has identified a lack of consistency across paired documents in the number and type of harms reported, approaches to code harm terms and the availability of contextual information required for interpretation. Our results are in line with other studies that found a high number of discrepancies in ADRs between the USA and Europe across a range of differing therapeutic areas.7–9

The lack of consistency could be in part due to differing rule-based arbitrary criteria used to identify ADRs from all AEs reported in a trial as these criteria even varied within a document. It was not possible for us to assess whether the trials contributing to both paired documents were identical, potentially adding further discrepancies. It is possible that the year of approval may be associated with lack of consistency; however, with only a few drugs included and low consistency across all drugs, we did not explore this aspect, but previous studies have examined this and found conflicting results. Eriksson et al7 reported that drugs approved after 2000 showed higher consistency, but Kesselheim et al8 reported that length of time on the market was not associated with consistency.

The lack of consistency demonstrated in this study across matched drug documents raises questions as to whether information across a class of drugs produced by different manufacturers could ever be usefully compared. Despite this, risk-benefit comparisons, in which these documents play a fundamental role in informing prescribers knowledge of harm, are undertaken daily.

The differences in the number of ADRs found between paired documents of the same drug may not all be real differences in the harm profile, but could simply be differences in ADR coding and presentation. It has been shown that ADR summaries are dependent on the dictionary used and even when the same dictionary is used there is inter-observer variation due to coding conventions.19–21 With differing dictionaries used to code harm, data aggregated to different levels and lack of procedures reported, it was not possible for us to draw further conclusions within this study.

In general, the US documents contained more information on methodological aspects and contextual information needed to assess and interpret harm events, such as data source of ADR and information by indication. However, the additional volume of material made the overall harm profile harder to absorb than information in the European documents. This contrasting approach of reporting ADRs in either insufficient or excessive detail has previously been raised by Kesselheim et al.8 The guidance from the EMA and FDA state that harm information should be ‘clear and readily accessible’.5 ,6 Better ways to prioritise and present harm data are required to enable patients and prescribers to undertake informed risk-benefit decisions.

Current clinical practice may often be too reliant on prescribers drawing on personal experience, knowledge from a colleague or abbreviated ADR summaries available in prescribing handbooks. Most of the product documents included in this review reported more than 100 ADRs, with one document listing over 400. Studies in different therapeutic areas have also reported similar high numbers, with Kesselheim et al8 reporting a median of 105 ADRs for 20 top-selling drugs across four countries, Eriksson et al7 reporting 4003 ADRs in 40 drugs across two countries, and Duke et al22 reporting a median of 49 ADRs across 5602 drugs and 10% having greater than 150 ADRs reported. With such large numbers of ADRs reported and no information provided on severity, duration or recurrence, it is questionable how useful the information in these documents is. In general, the US documents contained a larger number of ADRs and it was often unclear if these harm events were truly ADRs rather than AEs. It is possible that the over-inclusion of many AEs is made to minimise the risk of litigation since the manufacturers of the brand drug are held legally responsible if these are not reported.23 However, the over-inclusion of AEs will make the absorption of pertinent harm information harder for prescribers and patients. More rigorous guidance underpinned by empirical evidence is needed to reduce current uncertainty around what ADR information should be included and how it should be summarised and presented.

The wide risk categories predominantly used in the European documents should be reviewed. Adverse reactions are placed into categories such as ‘very common’ (>1/10) or ‘common’ (>1/100 and <1/10). While the purpose of these risk categories may be to assist readers in assimilating a large volume of harm information, they are too vague to be useful for serious ADRs. ADRs such as angle-closure glaucoma, rhabdomyolysis, ataxia and arthralgia are categorised as ‘common’, implying the risk could be anywhere from 1 in 100 to 1 in 10. It is difficult to see how such an estimate can be useful in informing a risk-benefit decision especially in the absence of any information on severity, duration or reversibility. Instead, it may be preferable, where possible, to provide point estimates of risk with an indication of precision for life-threatening events or events that significantly impact a patient's quality of life. In contrast, wide risk categories for ADRs with minimal impact on quality of life or physical health, such as ‘yawning’, may be sufficient.

All antidepressant documents included a warning for suicidal events with anticipated time to onset. No other information regarding onset, duration or recurrence was included for any other ADR. A patient may be willing to risk the occurrence of tinnitus or peripheral vision loss if the condition resolves but not if the duration is excessive and recurring.

Current guidance for clinical trials in the published literature has existed since 2004,24 but has made little impact on the quality of harm reporting.1–3 This present review also found that data from trials were inadequately summarised and reported, and, notably, differed across paired documents of the same drug. Aagaard and Hansen9 found that the sources of discrepancies across documents were primarily from information based on clinical trials.

Bringing together many sources of disparate information from various locations is technically problematic. An ambitious new initiative with the aim of collating information from a variety of sources (spontaneous reports, published literature, product documents and observational data) has been launched, but it is still in its infancy and its effectiveness is as yet unknown.25 There are initiatives to rank drugs for risk of a specific drug harm such as drug-induced arrhythmias (http://www.CredibleMeds.org, http://www.BrugadaDrugs.org), but these are lacking an overall harm profile. While the product information document is written with a focus on providing information to healthcare workers, we believe this publically available document is uniquely placed to be an invaluable source of information for patients and researchers. It is the only source that contains unpublished evidence held by the pharmaceutical company and includes all available evidence from clinical trials, post-authorisation safety studies and spontaneous reporting. What ADRs should be included and how to usefully present these multidimensional data to be informative to patients and prescribers requires further consideration.

Several studies have reported that patients would like more information on ‘side effects’, believing that full disclosure of side effect information would help them make more informed treatment decisions.26 ,27 Many studies have shown that giving patients more information does not lead to an increase in the reporting of ‘side effects’.28 ,29 A recent review showed that patients have a tendency to underestimate their risk of harm.30 But there is evidence that when patients have been presented with harm leaflets that lack numerical information, they were more likely to overestimate their risk.31 ,32 It is a statutory requirement for all drug manufacturers to include Patient Information Leaflets (PILs) as an insert in the medicine package. A review of the effectiveness of PILs found that patients thought them to be poor quality in terms of layout and content, and considered that they did not help increase knowledge to allow informed risk-benefit decision-making.33

While it is clear that all serious and unexpected ADRs should be reported, it is also important to know the true risk of common and anticipated ADRs, as they have cost-benefit consequences and can significantly impact on a patient's quality of life. In a trial, many hundreds of AEs may be recorded and a decision of what should be reported has to be made. Currently, common harm outcomes are usually selected by means of an arbitrary rule-based criterion such as ‘an AE that occurs in at least 2% of patients and is twice the placebo rate’. Using this type of criterion leads to flagging the most frequent but not necessarily the most important ADRs from a patient or healthcare provider perspective.

It has previously been proposed that consistency in harm information from trials in the published literature could be improved by developing a set of ‘core outcomes’ to be reported by drug class based on the principle proposed for effectiveness studies (http://www.comet-initiative.org).2 These ‘core outcome sets’ are the agreed minimum outcomes that should be reported, and are developed through a consensus process that includes all key stakeholders. A particularly common issue pertinent to harm profiles concerns the absence of information for a particular harm. It can never be clear whether this is simply because it was not reported (below the threshold of a rule-based criterion) or no events were observed. To reduce the inconsistency demonstrated in this review, we would endorse development of core ‘harm’ outcome sets by drug class to be reported in addition to all serious and unexpected ADRs.

How harm data are collected has an important impact on the type and the number of events recorded.17 ,24 While we have not specifically explored this issue in the present study, it is clearly vital, and more attention should be paid to methods used to collect and record AEs.

This review highlights a number of weaknesses in harm information contained in product descriptions that are also relevant to harm reporting in primary studies. We propose a conceptual framework to address these issues in figure 1.