Results 40% (95% confidence interval 33% to 46%) of the press releases contained exaggerated advice, 33% (26% to 40%) contained exaggerated causal claims, and 36% (28% to 46%) contained exaggerated inference to humans from animal research. When press releases contained such exaggeration, 58% (95% confidence interval 48% to 68%), 81% (70% to 93%), and 86% (77% to 95%) of news stories, respectively, contained similar exaggeration, compared with exaggeration rates of 17% (10% to 24%), 18% (9% to 27%), and 10% (0% to 19%) in news when the press releases were not exaggerated. Odds ratios for each category of analysis were 6.5 (95% confidence interval 3.5 to 12), 20 (7.6 to 51), and 56 (15 to 211). At the same time, there was little evidence that exaggeration in press releases increased the uptake of news.

We aimed to clarify how often news contains claims or advice from health related research that go beyond those in the peer reviewed journal articles, and to identify the likely source of these exaggerations (press releases or news). Furthermore, we tested whether exaggerations in press releases were associated with a higher likelihood of news coverage, compared with press releases without exaggeration.

Previous research suggests that press releases can be a source of misinformation. Of 200 randomly selected medical press releases in 2005, 29% were rated as exaggerated and less than half provided appropriate caveats to their claims. 20 In a study of 23 press releases and 71 associated news stories about cancer genetics, two thirds of claims in the press release were at least as deterministic as the claims in the news. 21 However, since these studies did not compare press releases with statements made in the abstracts or discussions of the associated peer reviewed journal articles, they may not be examples of exaggeration beyond what journal articles routinely include themselves. Indeed, in a study on “spin” in the reporting of randomised controlled trials (70 press releases and associated journal abstracts, 41 news stories), in only four cases the news contained spin where the associated journal abstract did not. 22

“Information subsidies” such as university press releases have long been used to deliver salient aspects of selected research, 12 13 and as journalists are increasingly expected to produce more copy in less time 14 15 these press releases have become the dominant link between academia and the media. 16 17 As such, information included in press releases is highly likely to be included in news stories. 18 Although accurate information, alone, is not sufficient for clear public understanding and informed behaviour, 19 it is nevertheless important that health and science news is not misleading, especially when it includes health advice for readers. News pieces have a different purpose to, and readership from, journal articles and are not expected to reproduce them or express claims in the same way. However, given that news is often explicitly or implicitly blamed for distorting and exaggerating scientific findings, 9 it is pertinent to determine the sources of such misreporting. In fact there is little evidence on how often news stories go beyond what scientists state in peer reviewed journal articles, and, when they do, whether misrepresentation is already present in the un-peer reviewed sources supplied by scientists and press offices.

The framing of health related information in the national and international media, and the way in which audiences decode it, has complex and potentially powerful impacts on healthcare utilisation and other health related behaviour in many countries. 1 2 3 4 5 6 The media also demonstrably influences the behaviour of scientists and doctors. 3 4 Such impacts may often be beneficial, but misleading messages can have adverse effects (even if these effects may be difficult to predict and prove because the responses of audiences are complex and multiply determined). 6 This problem is not restricted to rare dramatic cases such as vaccination scares 7 8 ; the cumulative effect of everyday misreporting can confuse and erode public trust in science and medicine, with detrimental consequences. 9 10 11

Methods

From publicly accessible university repositories we identified all the press releases based on published studies with possible relevance for human health (biomedical and psychological sciences; fig 1⇓) issued in 2011 by the Russell Group universities (the 20 leading UK research universities). We selected these universities as a clearly defined group with international prominence; we did not expect differences between this sample and other UK or international press releases (see for example20 21). For each relevant press release (n=462) we sourced the associated peer reviewed journal article and print or online news stories (n=668) from national press using the Nexis database, BBC, Reuters, and Google (we did not include broadcast news; the number of news stories per press release ranged from 0-10). We coded each journal article, press release, and news set using a detailed protocol available online (http://dx.doi.org/10.6084/m9.figshare.903704; supplementary information SI sections 1-3 provide full details of our sample and methods). Each set took on average 3-4 hours to code. We double coded 27% of press releases and journal articles and 21% of news stories (concordance rate 91%, mean κ=0.88; given the large number our simulations show that 10% disagreement would not influence our conclusions, see supplementary section SI7).

Fig 1 Identification of press releases based on published studies with possible relevance for human health (biomedical and psychological sciences

Taking the peer reviewed paper as a baseline (which is not to assume that peer reviewed publications are true; many already contain exaggeration), we sought cases where news stories offered advice to readers, made causal claims, or inferred relevance to humans beyond (or different to) that stated in the associated peer reviewed paper. Given the likelihood that some statements in journal articles themselves would be considered exaggerated by other scientists in the specialty, our overall levels of measured exaggeration are likely to be underestimates. We then asked whether such discrepancies were already present in the corresponding press release. For example, if a study reported a correlation between stress and wine consumption and the news story claimed that wine causes stress, what did the press release say? Similarly, if a news story claimed a new treatment for humans but the study was on rodents, what did the press release say?

We focused our study on analysing advice to readers to change behaviour, causal statements drawn from correlational results (cross sectional and longitudinal observational data), and inference to humans from animal research.23 Explicit advice clearly has the potential to influence behaviour, as do causal claims about what factors influence health. It is notoriously difficult to ascertain cause from correlational results. For example, a correlation between consuming wine and a disease could occur because wine increases the risk of the disease, the disease increases the consumption of wine, or the consumption of wine correlates with another factor that is associated with the increased risk. For animal research, it is estimated that less than 10% of non-human investigations ever succeed in being translated to human clinical use.24 Over-selling the results of non-human studies as a promised cure potentially confuses readers and might contribute to disillusionment with science.11

Advice We coded each journal article, press release, and news story for the maximum level of advice it contained using four levels based on explicitness and directness: no advice, implicit advice (for example, “these findings suggest that mid-late childhood may be the best bet for childhood obesity prevention”, “simply exercising with a best friend or having a friend who is a good exercise role model increases the chance of a child keeping fit and active”), explicit advice, but not to the reader or general public (for example, “I think we now have enough evidence to say that pulse oximetry screening should be incorporated into everyday clinical practice”, “ambulatory monitoring is recommended for most patients before the start of hypertensive drugs”), and explicit advice to the reader or general public (for example, “children who are thirsty should be encouraged to drink water”, “for anyone considering taking aspirin I would recommend . . .”). Relevant samples for analysis of exaggeration of advice were those containing at least one implicit or explicit advice statement anywhere in the journal article or press release or news (n=213 press releases, n=116 press releases with news; n=360 news stories).

Causal statements from correlational results For journal articles, press releases, and related news stories associated with correlational results we coded for the strength of the main statements of the findings. For press releases and news we used the title and first two sentences as their main statements, since nearly all follow the “inverted pyramid” structure of stating their main claims first.25 For journal articles we used the abstract and discussion. We used a seven point scale to rate increasing levels of determinism, where the presence of stronger statements trumped weaker ones: no statement (in which case no further comparison was possible), explicit statement of no relation, correlational (for example, “drinking wine is associated with increased cancer rates”), ambiguous (for example, “drinking wine linked to cancer risk”), conditional causal (for example, “drinking wine might increase cancer risk”), can cause (for example, “drinking wine can increase cancer risk”), and unconditionally causal (for example, “drinking wine increases cancer risk”). For analysis of causal claims we focused on correlational research, which we defined as observational cross sectional and longitudinal designs. We did not analyse qualitative, interventional, or simulation designs. We coded the first claim statement for our primary analysis (relevant samples for analysis were 182 press release, 95 with news; 261 news stories). Where a second statement occurred about a different variable pair, we also coded these for replication (see supplementary section SI5 for analysis).

Conclusions for humans from studies in non-humans For each non-human study (animals, cells, or simulations), we coded whether the main statements of press release and news were phrased as explicitly non-human, implicitly human (for example, “a pregnant mother’s stress level affects the brain of her unborn baby”), or explicitly human (for example, “a pregnant woman’s stress . . .”). For journal articles we searched the discussion section and abstract for any statements about human relevance. Relevant samples for analysis were 105 press releases, 48 with news; 115 news stories.

Caveats and justifications We searched the whole press release and news stories for any caveats stated for the advice, causal claims, or inference to humans (for example, “This is a population study. It cannot say definitively that sugary drinks raise your blood pressure, but it’s one piece of the evidence in a jigsaw puzzle”, “The scientists who carried out the study emphasized that they could not say for certain . . .”). Similarly, we searched for justifications of the advice, claims, or inference (for example, “even after taking into account the effect of extra body weight on blood pressure, there was still a significant link with sweetened drinks”).

Study facts and quotes We also coded facts about the study and press release, including sample size, duration, completion rate, and the source of quotes. These are analysed in section SI11 of the supplementary file. Further details of the coding methodology are given in section SI2 of the supplementary file. All coding sheets (n=462), full instructions for coding, and data analysis files and programs are available online (http://dx.doi.org/10.6084/m9.figshare.903704).