“IF WE didn’t take any risks, we wouldn’t approve any drugs,” says Susan Ellenberg, a professor of biostatistics at the University of Pennsylvania. “Some people will always want a new drug sooner and say they’re willing to take a chance. Others will ask, why didn’t you study it longer and find out about this horrible side-effect?”

During her long career, Dr Ellenberg has used data to quantify and communicate those risks. Along the way she has helped to shape a discipline that owes as much to ethics and philosophy as it does to pure mathematics. Now medicine is entering a new digital age, one of Big Data and high-tech personalised treatments that are tailored to an individual’s genetic make-up. But more data does not necessarily mean better data, so amid the increasing complexity it will be as important as ever to measure correctly which treatments work and which do not.

It is a job Dr Ellenberg is well suited to. She has already played a big part in improving the data-monitoring committees that now oversee virtually all clinical trials; she has helped establish standard practices for tracking dangerous treatments; and she has encouraged patient lobbies to find a voice in clinical testing.

But Dr Ellenberg nearly missed becoming a statistician at all. As a high-school maths teacher in the 1970s, she took a summer job analysing clinical trial data. Luckily, she became so engrossed that she quit her job and returned to graduate school for a doctorate in statistics. The basics of randomising subjects into different groups and leaving the patient (and ideally health-care workers as well) unaware of the treatment each was receiving were well known. However, there were still plenty of mistakes being made.

“In the old days, people used to throw out some of their data,” says Dr Ellenberg. “If a patient didn’t comply with their treatment, the researchers would say, how can they possibly contribute to the question of how that treatment works? So they just dropped them.” In one case Dr Ellenberg worked on in the 1970s, doctors wanted to test whether chemotherapy could help people recovering from colon cancer surgery. The study required patients to start chemotherapy within six weeks of their operation for the best chance of catching any remaining cancer cells. Those who missed the deadline were automatically excluded from the analysis.

Dr Ellenberg realised that most reasons for starting treatment late, such as a slower recovery from surgery because of old age or a particularly large tumour, would probably mean a poorer prognosis regardless of any subsequent treatment. Excluding those people would leave the chemotherapy group with healthier members on average, making a drug look beneficial even if it did nothing. Dr Ellenberg insisted that the investigators track everyone who had been randomised into the study, even if they were treated late or not at all.

In 1988, Dr Ellenberg became the first chief of biostatistics for AIDS at the US National Institute of Allergy and Infectious Diseases. She arrived at a desperate time. HIV appeared to be a death sentence, patients were demanding treatments, however unproven, and doctors were struggling to catch up. With most infectious diseases, patients could be treated and followed up within weeks to see whether the pathogen had disappeared. With HIV/AIDS, they might have to monitor trial members for years to see who lived and who died.

Measuring surrogates

Dr Ellenberg championed a concept called surrogate endpoints that she had pioneered in cancer trials. These are biochemical measures that can indicate quickly whether a patient in a trial is likely to improve, remain stable or deteriorate in the long-term. For example, blood pressure can be a surrogate endpoint for cardiovascular mortality. The challenge with AIDS was working out which of dozens of biological markers had the best predictive value. Dr Ellenberg helped narrow these down to ones that were strongly associated with long-term survival, such as CD4 white-blood-cell counts. “I wish I could tell you that led to wonderful results and now we know how to do it,” she says, “But we’re still limping along.”

The problem is that a surrogate for one treatment may not work with another, either because the second treatment functions differently or has side-effects. But it was still a step forward, allowing investigators to screen potential drugs more quickly. Nothing could be fast enough for some activists, however, who wanted early access to anything that might slow the progression of AIDS. “The clinical leadership was unwilling to talk with activists at that point,” says Dr Ellenberg, “But I saw that the Act Up group in New York had a very carefully thought-out set of principles for doing AIDS trials.”

Dr Ellenberg welcomed Act Up to her statistical working group on AIDS, and changes began to trickle through. Until then, some studies had not allowed trial patients to take drugs other than the one being tested, even though many AIDS sufferers needed a cocktail of medications to fight opportunistic infections. Dr Ellenberg showed that a study could deliver useful results while allowing its members to continue with life-saving medicines. Patient groups are now routinely involved in planning clinical trials.

The role of placebos in clinical testing was a thornier problem. The most reliable results can always be obtained by comparing two identical groups, one of which receives a treatment and the other an inert placebo. Ethically, however, doctors are loth to withhold an effective treatment where one exists, so many trials simply compared a new drug to an existing one. In 1993 Dr Ellenberg moved to the US Food and Drug Administration (FDA). In a series of scientific papers, she and a colleague demonstrated that such trials can often fail to demonstrate the effectiveness of new treatments. In 2002, the World Medical Association changed its recommendations to permit placebo-controlled trials explicitly where patients would not suffer serious or irreversible harm.

Without the right analytical methods, more data just gives a more precise estimate of the wrong thing

The same year, Dr Ellenberg wrote a book about the importance (and the dangers) of analysing data as it accumulates during a clinical trial. Her ideas of how data-monitoring committees should function quickly became standard practice. It had long been realised, for example, that a trial might reveal one treatment to be much better than another. The only ethical thing to do in that case would be to stop the trial and give everyone the superior drug. In the past, statisticians keen to find such magic bullets would crunch their data every few weeks or months. “But if you look at your data often enough, sooner or later you’ll observe by chance that one arm of the test looks better,” says Dr Ellenberg. “There is now a mistrust of the whole concept of early termination.”

She also cautions against the temptation to set statistics aside when faced with something that appears to be urgent: “There are groups saying they would be opposed to doing randomised trials for drugs or vaccines for Ebola because it’s so serious. But we’re not doing anybody any favours if we don’t find out whether these drugs or vaccines actually work.”

Much of Dr Ellenberg’s work at the FDA focused on the safety of medicines, particularly vaccines, once they were on the market. No clinical trial can ever catch the rarest side-effects but tracking those down from sporadic reports, anecdotes and coincidences is incredibly difficult. She notes that most infants are vaccinated and sometimes children get very sick. But is it the vaccine or just coincidence? “I was trying to make something out of the worst, dirtiest kind of data that you could possibly imagine,” adds Dr Ellenberg.

The arrival of electronic medical records and the advent of Big Data promises massive statistical analyses that can uncover everything from uncommon side-effects to how peoples’ genes might affect their future well-being. The technology is likely to be particularly useful in detecting bad treatments, thinks Dr Ellenberg. While most reported problems may continue to be coincidences, at least biostatisticians will be able to compare reliable lists of who took a drug and who experienced unpleasant reactions. The problem, says Dr Ellenberg, is detecting the signal from the noise. “The more people you have the richer your database will be but also the more ways there are to be misled by the data.” Without the right analytical methods, she believes, more data just gives a more precise estimate of the wrong thing.

From the genes

Dr Ellenberg points out that services like 23andMe, which provide ancestral and medical interpretations of individuals’ genetic information, have not yet delivered the revolution in health that many had expected. In the early days of genomics, excited mathematicians thought they had discovered thousands of correlations, most of which were chance findings. Dr Ellenberg also worries that presenting people with links between particular genes and health outcomes might lead them to worry needlessly or seek out potentially harmful treatments for conditions they do not yet suffer from.

In his state-of-the-union address, Barack Obama lauded personalised medicines. But these are tricky to approve. When a disease affects millions, large clinical trials can reliably spot even small differences between drugs. But for personalised treatments, or ones targeting rare “orphan diseases” that affect only a few people, those differences become much harder to spot. Nevertheless, Dr Ellenberg believes statistics can help by integrating evidence from other trials.

Dr Ellenberg continues to work on surrogate endpoints and clinical trials, including a new study testing an innovative approach to attacking HIV. She also recently travelled to Botswana to help statisticians and clinicians there develop their own biostatistics programmes. Like most medical academics, Dr Ellenberg would like to see an end to the practice of some pharmaceutical companies quietly burying trial data that is inconvenient to them. Thousands of clinical trials have never been registered with oversight agencies and results from around half of all clinical trials (often those with unfavourable results) remain unpublished. Making that data available to statisticians would almost certainly lead to new discoveries and clinically useful findings.

However there could also be negative consequences. “Sharing raw data could promote inappropriate re-analyses,” warns Dr Ellenberg. She says there are many who would be ready to believe any analysis claiming to prove that vaccines caused harm.

That the dry world of statistics is becoming a battleground of ideas and commercial interests, affecting the future of medical care and the lives of people around the world, may shock some. For Dr Ellenberg, who has spent her professional life emphasising the life-saving importance of accuracy, it is no surprise at all. “We’ve got all this data,” she says. “The answer isn’t to ignore it. The answer is to figure out how to limit the number of mistakes we make.”