“THIS is undoubtedly the biggest advance in mortality measurement in four decades!” Thus Alan Lopez of the University of Queensland, in Australia, trumpeted the virtues of a study on trends in infant death that was presented to a gathering of health experts in Washington, DC, this week. Though Dr Lopez is not exactly a neutral observer, as he was both a participant in the study and one of the authors of the resulting paper, which was published on May 24th by the Lancet, his hyperbole is excusable.

The paper proclaims good news on three fronts. It concludes that there are around 1m fewer deaths each year of children under the age of five than previous work had indicated (see chart). It advances statistical techniques that appear to be better than earlier approaches at exposing trends in mortality, especially when the underlying data are dodgy. And its authors suggest a way of analysing data that seems a robust alternative to the top-down, politicised approach typically employed in the bureaucracies that make so many of the decisions to do with global health.

So were those authors hailed as conquering heroes at the conference in Washington? Not quite. The room was full of experts from precisely the sort of health bureaucracies, ranging from United Nations agencies and government ministries to their myriad consultants and hangers-on, that the authors of the paper criticise. Not all of the gathered panjandrums were won over by the study's claims of fewer deaths, better methods and more openness. Take these in turn to see why.

Totalitarians, regress!

The most important claim is that the number of infants dying each year around the world has dropped from 11.9m in 1990 to roughly 8m in 2008 (and is projected to be 7.7m in 2010). That contrasts with a study released last year by UNICEF, the United Nations' children's fund, which estimated that the fall was from 12.5m in 1990 to 8.8m in 2008. Those 8m deaths are still far too many, of course, but it is surely cause for celebration that infant mortality appears to have fallen by over a third during the past two decades. Even in sub-Saharan Africa, say the authors, “there is evidence of accelerating declines from 2000 to 2010 compared with 1990 to 2000.”

And health officials and activists did celebrate the news of fewer deaths, at least publicly—but some are quietly grumbling. They worry that this revision of the statistics will lead a confused public to question all mortality data, as happened when the numbers for AIDS were revised downward in 2007. They fret that the problem will thus seem less of a crisis and that this will make it harder to raise money.

The second reason for cheer offered by the paper—superior statistical methods—is also challenged by some. The underlying data from many countries are missing or unreliable, so it is accepted practice for number-crunchers to use computer modelling to fill in the blanks. However, the Institute for Health Metrics and Evaluation (IHME) at the University of Washington, which is home to several of the paper's authors, argues that the methods used by official agencies are antiquated and flawed. Instead, the authors of the new paper employ a technique called Gaussian process regression (GPR), which is not commonly used in the field of global health, but which they believe is more accurate, to plug the gaps and make the forecasts.

Predictive modelling will never be as valuable as lots of boots on the ground collecting reliable data, of course. And proud as he is of GPR, Christopher Murray, the head of the IHME, acknowledges that it accounts for only a third of the explanation for his study's striking results. Better data, he says, make up the other two-thirds of the story. For one advantage the authors had over their predecessors was the fruits of a flurry of investment in measuring mortality that was spurred by the promulgation in 2000 of the UN's Millennium Development Goals. This is a set of objectives for the reduction of poverty and its associated ills, that UN members have agreed to try to achieve by 2015.

But how good are those data, really? Kenneth Hill, a demographer at the Harvard School of Public Health, applauds the “impressive job” done by the paper's authors in compiling information from diverse sources. However, he is troubled that much of the drop in mortality recorded is the result of changes in the figures for a mere half-dozen countries. These include Afghanistan, Angola, Congo and Nigeria, which Dr Hill suggests produce health data that are, at best, inconclusive. He argues that “to state conclusions so strongly, and to criticise other agencies so forcefully, a much stronger evidentiary base is required.” Dr Murray, however, insists the data from the countries in question are valid, and so too are his paper's conclusions.

New model army

And what of the fancy statistical tools? Eduard Bos, a population expert at the World Bank, is unimpressed. He recalls that the IHME team made similarly grand claims when its previous study on child mortality was unveiled in 2007. That work also used a novel statistical approach (known as the Loess method) to analyse data, but he points out that its conclusions now appear even more incorrect than rival studies done by UN agencies. “Modelling is extremely unlikely to be the answer to better data,” he concludes.

Nonsense, says Dr Murray. He thinks there is a methods revolution under way in the field, but argues that most of the statistics experts at official agencies and governments “are 20 years out of date”. As the former data guru for the World Health Organisation, he accepts that he is also guilty of this sin (he did not even know what GPR was until a few years ago), but he now compensates for the omission by hiring young statistical wizards.

Julie Rajaratnam, one of these wunderkinder and the lead author of the new paper, says her team tested GPR's predictive value in various ways—by, for example, withholding from the analysis the last ten years of data in every country with at least 20 years of data, and by withholding 20% of the surveys of individual countries—in order to see how well the missing values were predicted. On every measure, the Gaussian approach proved better than the alternatives.

That exercise points to the most provocative aspect of this paper. The IHME believes that scientific journals and official agencies should adopt rigorous vetting methods of predictive power, such as the 20% tests and decadal-withholding exercises. This flies in the face of the traditional approach by which most UN agencies develop the official numbers for indicators of health in consultation, often secretive, with member governments.

That helps explain why IHME, since its founding in 2007 with money from the Gates Foundation, has met with bitter resistance from the old guard of public health. The new paper has already offended so many people that the Lancet plans to run half a dozen letters carrying “outrageous critiques” of it in early June.

In response, some of the speakers at the conference labelled the global health-information bureaucracy “totalitarian”, and only partly in jest. Dr Murray argues that the best data and analyses must be embraced, even when they are inconvenient, if policies are to be shaped properly and resources allocated sensibly. And this is what the study's authors think they have done. He observes, for example, that the conclusion that child mortality is falling fast in many parts of sub-Saharan Africa argues for more, not less, money for the successful programmes in those countries. As the old saying among bureaucrats goes, he who controls the numbers commands the power. With luck, that control is passing to people who are getting the numbers right.