We’re hardwired to focus on bad news stories, but that is not the whole truth. Ed Cumming meets the optimistic statisticians and economists using facts to reveal why more people are healthier and happier than ever before

Not every problem has an obvious solution, which is why during the 1850s Britain bought 300,000 tons of bird poo a year from Peru. This was guano, the wonder fertiliser that had been discovered by Europeans at the start of that century. It was shipped back to the motherland, where it helped to feed the burgeoning and rapidly industrialising population, mainly through the medium of turnips.

In a modern globalised world, the idea of transporting large quantities of avian dung thousands of miles in wooden sailing boats to grow turnips seems less incongruous. For Ruth DeFries, a professor of ecology and sustainable development at Columbia University in New York, the guano craze is one example of how over the centuries human ingenuity has risen to the challenge of feeding ourselves.

“We have constantly figured out ways of keeping our soil fertile: recycling waste, transporting guano, digging up phosphate, collecting bones and finally synthesising nitrogen,” says DeFries. In medieval times, an acre of wheat would yield around 10 bushels. By 1950 it was still less than 20. Today it is more than 40.

As you might guess from the subtitle of DeFries’s 2014 book The Great Ratchet: How Humanity Thrives in the Face of Natural Crisis, she is hopeful about our ability to avert disaster. She argues that history has progressed by “ratchets”, where things improve; “hatchets”, where unforeseen problems occur; and “pivots”, where new solutions are found.

Facebook Twitter Pinterest Loaded to the gunnells: an illustration of boats shipping guano in the 1850s from Peru. Photograph: Universal History Archive/Getty Images

“We’re in a moment where we have had a big ratchet over the past 50 years,” DeFries says. “We’re now seeing the hatchets fall: environmental problems, climate change, nitrogen run-off, inequitable distribution. Hopefully we’re in the pivot part of the cycle.” Greater efficiency in fertiliser and water use, and possibly shifts in our diet, will help us continue to stave off the long-heralded Malthusian catastrophe.

DeFries is not the only academic trying to tell a more positive story with data. We are hardwired to seek out bad news and focus on the things going wrong. We’re on the edge of our seats, secretly waiting for calamity. Usually the news provides.

“Things that happen in an instant are mostly bad,” says Max Roser, an economist at Oxford University. “It’s this earthquake or that horrible murder. You’re never going to have an article on the BBC or CNN that begins by saying: ‘There’s no famine in south London today’ or: ‘Child mortality again decreased by 0.005% in Botswana’. But once you turn to statistics it gets much harder to have a pessimistic story.”

In addition to his public events and lectures, Roser runs Our World in Data, which shows how standards of living have changed over the centuries. Begun in 2011 as a “massive procrastination exercise when I was trying to write a book”, as Roser says, the site now employs full-time researchers and is looking for new sources of funding. Using the best and most official data available, he shows how global poverty continues to fall while standards of living, health and education continue to rise.

Facebook Twitter Pinterest Look and learn: an example of the everyday ‘good news’ we so rarely hear. Education levels are soaring in the developing world. Photograph: Alamy

He’s not alone. In the UK, Matt Ridley has been beating his Rational Optimist drum for years, while Harvard professor Steven Pinker argued persuasively in his book The Better Angels of Our Nature that violence is on the decline. Presiding over the field is Hans Rosling, the Swedish professor who is the closest thing statistics has ever had to a rock star. His TED talk The Best Stats You’ve Ever Seen has been viewed more than 10m times. Last month the BBC aired a lecture, timed to coincide with new UN development goals (and made with input from Roser), called How to End Poverty in 15 Years. Rosling lectures all over the world to rapturous audiences, making his points with humour, striking visuals and the occasional flash of temper with interviewers who don’t get it. To watch him in full flow is to feel, in a funny way, proud to be human.

Except if you are his son, Ola, in which case it can be irritating to see Dad take all the credit for the family business.

“Hans gets all the fame and he does a very small part of the work, which is extremely annoying,” Ola says. “It is me who reviews all the statistics to make sure we don’t make any mistakes with the data. And our brand completely depends on not making mistakes.”

The brand is manifested in Gapminder, a public-access site founded by Hans, Ola and Ola’s wife Anna in 1999. At the site’s core is a customisable graph, where you can plot different data trends against each other and break them down by country. If you ever wondered how female literacy correlates with GDP, Gapminder is where you can find out. There is plenty of good news.

Facebook Twitter Pinterest On the up: Hans Rosling of Gapminder gives a talk. His TED talk, The Best Stats You’ve Ever Seen, has been viewed more than 10m times. Photograph: Cordon Press/Corbis

Not that you should put it like that to the Roslings themselves. “It’s not good news,” Ola says, when I suggest that this is the effect Gapminder – and Hans’s presentations – give the casual reader. “It’s just news. It’s factual news. It appears good because the default worldview has so much negativity. But sometimes the news is negative: look at the refugee crisis, climate change or inequality in the USA. We are trying to promote a state of mind called factfulness. This is a relaxing feeling of not carrying around opinions that often have no basis in reality. It’s very stressful to defend ideas that are not true.”

Not everyone is on board with the Gapminder project. Population Matters, a group concerned about rising global populations, has argued that the Roslings and their staff are not sufficiently open to the dangers presented by the earth’s estimated future population of 10bn. The Roslings and Roser are also up against the preconceived opinions of politicians, who twist the numbers to their own ends.

“Not everyone responds positively to the data,” says Ola. “Extreme libertarians who believe that we should have no government at all don’t like it, because it’s clear that good governance, and quite large governments, are really good for economic growth and for freedom. It’s the same with extreme environmentalists, those who believe that animals come first and perhaps some humans have to die. It’s about finding a compromise between society, the economy and the environment.”

Conversely, Roser worries that the fact that, by many measures of development, capitalist democracies have been successful, makes this long-term data analysis susceptible to hijack by free-market evangelists. But he agrees that part of the aim is to deflate misguided notions.

Facebook Twitter Pinterest Ruth DeFries: ‘We’re now seeing the hatchets fall.’ Photograph: Bruce Gilbert

“I just came back from the US, where I met this guy with a start-up. He was from a poor background but is now worth millions of dollars,” Roser says. “He was telling me how awesome the USA is compared with other places in the world because you have high social mobility and people can rise through the ranks. But that’s not right. He was very enthusiastic, but one of his core beliefs was just wrong. I told him that if you want the American dream then you’d better move to Sweden. He was really surprised.”

For the Roslings, the opening up of global data represents a revolution in social sciences. “It’s the first time we’ve been able to understand our own societies based on facts,” says Ola. “And we’re measuring ignorance in the public, to guide us to the most misunderstood facts. People have such a gloomy picture of the world: that there’s no democracy in Africa, for instance, which is absolutely wrong. Democracy in Africa has changed like crazy over the past 40 years. There has been an amazing improvement.”

Gapminder and Our World in Data are a treasure trove of surprising thoughts and facts. Artificial light produced by LED since 2006 is 13,500 times cheaper than produced by candlelight in 1300. By freeing up time for leisure rather than boring labour, the washing machine was arguably the greatest invention of the industrial revolution. At the start of the Vietnam War in 1964, the US and Vietnam had wildly divergent life expectancy and family sizes; by 2003 they were the same. Since 1960 Thailand has achieved improvements in child mortality rates that took Sweden 150 years. Old ideas of “western” and “developing” countries, or that it’s possible to think of sub-Saharan Africa as any kind of homogenous bloc, are completely debunked, even while they remain in circulation in our media.

Misconceptions exist closer to home, too. Take the subject of inequality, where it is tempting to lump the UK in with the US. “People make the mistake of framing the UK as a softer version of what’s happening in the US,” says Roser. “But it’s not true. Recently the very richest in the UK have had much more rapid income growth, and inequality grew sharply in the 80s. But since then there has been strong growth for the majority of people in Britain. In the US inequality has increased for three decades, with stagnating incomes for the bottom 50% or so.”

He believes that the solution lies in schools. “The kind of maths that people are taught at school focuses on algebra and calculus, which they hardly ever use later in life,” he says. “You use statistics all the time – for the weather forecast or calculating your income. And whether you’re talking about it with other academics or in the pub, these are topics that matter to people.”

Ola Rosling agrees. “It’s common among the highly educated to blame the mass media, but the mass media is just giving people what they want. We have to teach children how the media works.”

DeFries, Roser and the Roslings have a similar agenda: to show the world as it really is, and as a consequence what might be possible. “If people are bombarded with catastrophe, they stop paying attention,” says DeFries. “But the other side is that everything is fine and there’ll always be a techno fix. I wanted to find a way between those extremes by taking a historical look at how people have figured out how to produce food over many millennia.”

The guano craze didn’t last. Demand, dwindling supplies and Peruvian mismanagement saw the price rocket. The chemists of the industrial revolution came up with sodium nitrate, a much cheaper alternative, and by the mid-1880s the guano trade had all but disappeared. In the 20th century, a by-product of the First World War was synthesising ammonia. And so on.

DeFries grew up as environmentalism was taking off. “I put myself in the category of Earth Day Adolescent,” she says. But spending time in India convinced her that traditional activism was not the way forward. “I realised that the idea of keeping people from their basic needs in order to save the environment just isn’t a viable direction. People are part of the environment, and any consideration of the environment has to help them, too. I don’t want to say I’m wildly optimistic. But we’ve shown how ingenious we can be as a species. We have a pattern of solving problems.”