Ben Goldacre

The Guardian

Saturday 21 March 2009

Science is not difficult to explain. Today we will see how British journalists go out of their way to cherry pick which evidence they cover, and then explain the risks and benefits in what has been shown to be the single most unhelpful way possible.

“Screening all older men for prostate cancer ‘could reduce deaths by a fifth’” said the Mail. “Prostate cancer hope” said the Mirror. Calls for new policies on NHS cancer tests said the Independent. Prostate cancer screening could cut deaths by 20% said the Guardian. Better cancer screening is every man’s right was the editorial in the Scotsman, where they wound themselves into a froth of indignation.

But was this just British journalists finding something to complain about? Because all around the world, people were saying something completely different, on the same day, about the very same academic publication: Prostate Cancer Screening May Not Reduce Deaths said the Washington Post. Studies cast doubt on leading prostate cancer test said USA Today. PSA testing may not save your life, after all said Scientific American. Prostate cancer blood test does little to decrease death rate said the Sydney Morning Herald. And so on.

Why would the American and the Australian journalists say something completely different to the British ones, about the very same evidence?

Firstly, our journalists were simply confused. Not a single newspaper managed to clearly explain the risks and benefits of screening in the trial they were writing about. It’s very simple: the study took over 160,000 men between the ages of 55 and 69 and randomly assigned them either to get PSA screening, or to be left alone. The differences were marginal. Yes, there were 20% fewer deaths in the screening group. What does that mean in terms of real people, in real numbers you can understand, not percentages?

1410 men would need to be screened to prevent one death. For each death prevented, 48 people would need to be treated: and prostate cancer treatment has a high risk of very serious side effects like impotence and incontinence.

These figures are not hard to find: they are in the summary of the research paper.

For complex risk decisions like screening, it has been shown in three separate studies that patients, doctors, and NHS purchasing panels make more rational decisions about treatments and screening programmes when they are given the figures as real numbers, as I did above, instead of percentages. I’m not saying that PSA screening is either good or bad: I am saying that people deserve the figures in the clearest form possible so they can make their own mind up.

Journalists meanwhile – professional communicators no less – all used impenetrable percentages, called “relative risk differences”, or worse. The piece by the Guardian’s health correspondent quoted several entirely meaningless numbers. “214 prostate cancer deaths had occurred in the screening group and 326 in the “control” group.” Without explaining the other numbers in each group, these digits are hardly more than decorative. It went on: “The rate of overdiagnosis – defined as diagnosis in men who would not have clinical symptoms during their lifetime – was as high as 50% among those who were screened.” I cannot see how this number helps me understand the risks.

But it gets worse. British journalists also deliberately ignored one whole half of the research, and I’ll confess I’ve slightly lost my sense of humour over this. There were in fact two large studies on PSA testing published in the New England Journal on the 18th of March 2009, not one. They were both published on the same day, in the same journal, they are side by side on the same contents page. British journalists discussed only one of them: the one that said PSA screening does reduce deaths.

The study they ignored was huge too: it took over 75,000 men and randomly assigned them to either a screening programme, or no screening. It found no difference in death rates between the two groups at all, and in case you think it was a close thing, in fact, there was a non-significant trend towards more deaths in the screening group. Not one UK newspaper mentioned this trial.

So newspapers ignore one half of the evidence, and they fail to explain the other half properly. In the past, nobody could catch them, and nobody could compete with them. That has now changed. Anyone can write, and publish online, and appear in Google news alerts: the NHS, medical research charities, individual academics, journals’ press offices.

These people are intelligent, informed, and they can explain things clearly, without worrying about eyecatching hysteria, or space, or hyperbole. Some will be silly, some won’t be. If they ignore half the evidence, they will be busted in the comments, mocked, and sensible visitors will never come back. They can also link directly and transparently to scientific papers, which mainstream media still refuses to do. Journalists insist that we need professionals to mediate and explain science. From today’s story, their self belief seems truly laughable.

References:

Andriole, G.L. et al. Mortality Results from a Randomized Prostate-Cancer Screening Trial. N Engl J Med NEJMoa0810696(2009).doi:10.1056/NEJMoa0810696

Schroder, F.H. et al. Screening and Prostate-Cancer Mortality in a Randomized European Study. N Engl J Med NEJMoa0810084(2009).doi:10.1056/NEJMoa0810084