If aid organisations could see impending disasters they could save more lives. How can ‘superforecasting’ help?

The 'supers' who can predict the future: can we learn how to see disaster coming?

Tensions were growing in the run up to Burundi’s presidential elections in 2015. The announcement that President Pierre Nkurunziza would run for a third term – despite the constitutional stipulation that a president may only run for two terms – had triggered protests and violent clashes with armed police.



As the protests spread, the authorities shut down social media services and private radio stations and thousands began to head over the borders to neighbouring Rwanda. To in-country NGOs, the signs were clear that conflict would break out. And that, they knew, would lead to immediate humanitarian need.

Organisations began to alert the Start Network, a collection of NGOs who manage a multi-donor fund taking radical approaches to humanitarian work, about the prospect of a crisis. It seemed possible that, by anticipating the problem, they would have time to prepare and set up projects, including, for example, launching camps in the surrounding countries to host displaced people.

Responding only when the famine is declared is very, very ineffective Rudi Van Aaken, FAO

But at this point, according to Luke Caley, the Start Network’s crisis anticipation adviser, “the decisionmakers didn’t have the confidence or information” to release the funds to intervene. “It created more questions than answers,” he says, and by the time the organisations on the ground submitted a second alert and the funds were activated, the conflict had already begun. “It showed that we needed a system around this.”

Slow response costs lives

History shows time and time again that a slow response by NGOs and agencies can be devastating. As a result of the famine that hit Somalia between 2010–12, 260,000 people died yet humanitarian assistance to the country had decreased between 2010 and 2011. Rudi Van Aaken, the deputy head of the UN Food and Agriculture Organisation (FAO)’s Somalia team, told the BBC in 2013 that half of the casualties there happened before it was even broadcast as a famine. “[The] main lesson learned is that the humanitarian community should be ready to take early action,” she said. “Responding only when the famine is declared is very, very ineffective.”

Facebook Twitter Pinterest A mother looks at her two-year-old son who’s been diagnosed with severe malnutrition in Somalia, currently on the brink of famine. Photograph: Andrew Renneisen/Getty Images

Slow response not only costs lives, but money: for example, in the 2004–05 Niger emergency [pdf] the World Food Programme (WFP)’s initial food deliveries in February 2005 cost $7 per beneficiary, but by August when the situation had reached crisis point and the needs were much more severe, the cost per head had risen to $23.

Having a little foresight, especially when it comes to humanitarian crises, could change everything – but is it really possible to anticipate disaster? This was the question the Start Network began to ask. The answer, it turned out, was more positive than they had hoped.

Foresight... it’s not a gift that’s bestowed upon special people, but is a skill that can be learned and developed

Caley, 37, a calmly spoken man who becomes animated by the idea of forecasting, admits that he has a thing for behavioural science. Together with Paul Currion, a Start Network consultant who first promoted the idea of forecasting tournaments for the humanitarian sector, he saw the potential of the work of a Canadian professor at the University of Pennsylvania. Philip Tetlock, a 62-year-old political science writer, is known for essentially undermining all professional experts after a 20-year research project in the late 1980s. He had assessed the accuracy of the predictions of thousands of pundits and analysts and found the average expert was no better at forecasting than a random guess (or a dart-throwing chimp).

But it was a more recent project that caught the eye of Caley and the Start Network. In 2011, Tetlock was involved in the Good Judgement Project, a four-year study of prediction. More than 5,000 volunteer forecasters were involved, making 1 million predictions on questions from the Syrian civil war to North Korean politics. The results were measured in Brier scores, which in a very rough sense shows how far away from the truth your prediction is. The top 2% for accuracy beat the average by 65%, and were reportedly 30% better at forecasting than US intelligence officers. The conclusion was that it’s not just about the “wisdom of the crowd”; it matters who is in the crowd.

Tetlock and his colleagues found that there is such a thing as foresight, and it’s not a gift that’s bestowed upon special people, but is a skill that can be learned and developed.

Those in the top 2% are known as the “superforecasters”, or “supers” as they like to refer to themselves in private circles.

Famine: what does it really mean and how do aid workers treat it? Read more

To obtain this apparent superpower does not take a PhD or an exceptionally high IQ; it takes a certain mindset. It’s not such an alien idea; indeed, all of us are forecasters. We are forecasting when we make decisions about whether to move job or house, for example. These decisions will be based on expectations about what will happen in the future.

But superforecasting takes this further. It requires open-minded thinking, applying a scientific method to study data, instead of trying to put a narrative on a situation. You keep up to date with current affairs, use Google, conduct thorough research over a long period of time. You have to seriously consider the possibility that your prediction might be wrong, and you adapt your prediction as you gain more information. Accountability is important – you don’t hide behind words like “could” or “might”; predictions are given in percentage of likelihood.

Superforecasters are smart people, but they could be anyone – like 56-year-old retiree Bill Flack from Nebraska, who studied physics in college and worked for the US department of agriculture. He told Freakonomics Radio he doesn’t often read a newspaper, and thought the Good Judgement Project might be “fun to try”.

“I believe it is possible to predict the future, at least in some situations and to some extent,” writes Tetlock in Superforecasting, “and that any intelligent, open-minded and hardworking person can cultivate the requisite skills” … much like those in the humanitarian community.



It was exactly what Caley had been looking for. He and Currion called them in, and began working with an advisory group of early warning experts called Forewarn, and superforecasters from the Good Judgement consultancy, to find a way to use these forecasting powers in humanitarian work.

Will President Assad of Syria be in power this time next year?

In Bonn in December, they set up a workshop and called in people from all sectors: academics, the insurance world, NGOs, DfID and the WFP – anyone keen to have a go at predicting the future. Michael Story of Good Judgement, who ran the workshop, says they looked at questions such as “Will President Assad of Syria be in power this time next year?” and discussed how to approach that kind of prediction. You do so using an “outside view”.

“Rather than thinking I’m going to zoom into the presidential palace in Damascus thinking is this guy going to stay, you look for comparison and zoom out as much as you can,” he says. “For example, how long does a typical civil war in the Middle East last, how long does a civil war last, how long does a dictator stay in power in these types of circumstances …”

The world is stuck in the past, Hans Rosling showed us the way forward Read more

It’s important to avoid pitfalls. “There’s a tendency that when things are difficult to imagine, you translate that into being less likely, which can be dangerous,” points out Story. “But often vague things cover a wide range of compasses so it becomes more likely because there’s less detail – the less detail you have, the more likely it is going to happen because you can apply it to lots of different situations.”

“We want to develop processes that will give us forecasts which we can use to inform our resource allocation. Whether we fund one crisis or another could be informed by this based on our assessment,” Caley says.

Tetlock uses an example of food distribution: “Rather than waiting for a failed harvest to lead to drought, to people extending a begging bowl to international governments, instead you pick up when the harvest is likely to fail and start intervening then.”

The theory and plan is that ultimately the Start Network will be able to look into the future – and then to release funding to agencies on the basis of forecasts. The Start fund, supported by UK Aid, Irish Aid, the government of the Netherlands and the European Union, can be released within 72 hours of an alert or request being submitted by network members.

“Our approach is to try and build and demonstrate a better way to do humanitarian response,” Caley adds. “This work hasn’t been done before.”

And if they get it wrong? “Risk is considered a lot,” says Johan Eldebo, senior humanitarian policy adviser at World Vision, a member of the Start Network who applied anticipatory response to the impact of the El Niño weather phenomenon in Sri Lanka. He says they “apply the no regrets method: if you prepare for disaster and it doesn’t happen, that prep work is useful for those communities. For example, if we put our tents and preposition food and water outside of Aleppo or Mosul, someone will be able to use that.”

Though there is an opportunity cost, the cost of lives and money, is far greater from late response. The question is whether the rest of the humanitarian community – and especially donors – are willing to adapt to this new way of thinking.

Join our community of development professionals and humanitarians. Follow @GuardianGDP on Twitter.