With its poor understanding of data, the recent report from Ban Ki Moon’s independent advisory group is not a practical roadmap

In August, UN secretary-general Ban Ki Moon named his independent expert advisory group, 24 experts tasked with providing recommendations on how best to use data to deliver the sustainable development goals.

In a press release at the time, Ban said: “The data revolution is giving the world powerful tools that can help usher in a more sustainable future. The recommendations of the group will be important inputs to the post-2015 debate and our efforts to shape an ambitious yet achievable vision.”

On 6 November, those recommendations were published in a report entitled A world that counts, a cleverly crafted motivational manifesto, but by no means a practical roadmap on how to apply a “data revolution” to the future development agenda.

I have previously written about this in more detail, but essentially, the report’s key weakness is that it conflates several terms, and assumes automatic relationships between things such as “counting” and “knowing”.

Using four Venn diagrams, I’ve tried to illustrate some of the main misconceptions.

Not everything that counts can be counted

Image 1

The report strongly suggests that everything that matters can be counted. We know that this is not true. If the guiding principle for the sustainable development goals is to make decisions as if everything can be counted, the end result will be very misleading.

Data is not the same as statistics

Image 2

The “data revolution” hype is just one of many places where the difference between statistics and data is misunderstood. Data is not the same as numbers. Data literally mean ‘what is given’, so when we speak of data we are talking about observations – quantitative or qualitative, or even figurative - that can be used to get information.

To keep talking about data when we mean statistics may sound better, but it only leads to confusion. The report (pdf) calls on the UN to establish “a process whereby key stakeholders create a Global Consensus on Data”. What is that supposed to mean? That statement is meaningless if you exchange the word “data” with “observations”, “knowledge” or ‘evidence’. It can, however, make sense if you talk about “statistics”.

International organisations do have a natural role when it comes to developing global standards for official statistics. Reaching a global consensus on how observations and evidence constitute knowledge is futile.



More data does not mean better decisions

Image 3 Photograph: Morten Jerven

The report also says demanding more data will lead to better decisions. That is a statement of belief, and not a theory of change. What is often thought of as “evidence-based policy” turns out to be ‘policy-based evidence’. In other words, the body of statistics we have today is a result of policy decisions made yesterday.

The point is that we do not know how changes in statistics will change policy, and that uncertainty should raise the following questions: how much we should invest in numbers and what type of metrics we should invest in?

There are other methods to knowing than through counting

Image 4 Photograph: Morten Jerven

The report says: “Whole groups of people are not being counted and important aspects of people’s lives and environmental conditions are still not measured” and then this: “Never again should it be possible to say “we didn’t know”. No one should be invisible. This is the world we want – a world that counts.”

I understand the enthusiasm, but I want to warn against hubris. This is certainly not the world I want but I also think it should always be possible to say: “We didn’t know”. Numbers, or the act of counting, does not guarantee objectivity nor does it always make us wiser. It is a testament to the richness of life, and the poverty of numbers, that all things cannot be counted.

A version of this piece was first published on Jerven’s blog.

Morten Jerven is associate professor at Simon Fraser University. Follow @mjerven on Twitter

Read more stories like this:

Join our community of development professionals and humanitarians. Follow@GuardianGDP on Twitter.