0:33 Intro. [Recording date: December 19, 2012.] Russ: This is scheduled to be the first release of 2013, which means that 2012 is over by the time you hear this. And I'd like to get some feedback from you there in the listening audience. Please send me an email listing your three favorite episodes of EconTalk for 2012. You can find all the 2012 episodes as well as all of our previous episodes going back to 2006 at http://www.econtalk.org/archives.html. Please send me an email at mail@econtalk.org and in the subject line say "Favorites." I'll tweet on them at @EconTalker, which is my Twitter name.

1:46 Russ: Morten, your book is a fascinating and I have to say, somewhat depressing, look at the quality of data in National Income Accounting and other areas of data collection at the national level, but particularly in Africa, which is the focus of your book. What's the main problem with calculating Gross Domestic Product (GDP) or growth rates in Africa? Why is that so difficult and why has that been so inaccurate? Guest: Well, the main problem when compiling GDP estimates at statistical offices in most African economies is that of data availability. These departments who are putting together the national accounts which produces the GDP estimates have very little information about the economy they are supposed to create the measure of. In particular they may have very little information about food production; they may have more information about export crops. They know a little bit about some manufacturing, some of the larger operators; they know about government activities, but there are huge gaps in the information relating to what we call 'the informal sector' or the unrecorded economy. And moreover, this problem of reaching a reliable or valid GDP estimate varies considerably from country to country and it also varies considerably across time. Some countries have very little data; some countries have more data; and if you look through history as well this problem has varied in intensity since the 1950s. So, that's basically what my book is about, to try to map how serious these errors are and how they vary and what to do about it. Russ: And these are data that are being used--they are used for all kinds of reasons. They are used to try to determine how much aid to give; they are measures of internal value to the country. But they are often used by economists in the development field to assess whether certain policies are working or not working or at least whether certain countries are making progress or not making progress. And when you say they differ--these estimates vary by time and space, meaning across country and even within a country across time--that's very disturbing for the people who are running sophisticated statistical models trying to assess the effectiveness of various policies. Right? Guest: That's right. Let me just talk through a couple of examples or debates where GDP statistics are particularly important. One is, for instance, whether a country is ranked as poor or a middle-income poor as a country, according to the World Bank. If it is ranked as a poor country, such as Tanzania and Kenya are, then it's eligible for concessional lending through the International Development Association (IDA), the concessional arm of the World Bank. If it is a middle-income country, it is not eligible for that kind of lending. And to take another example, it was in Ghana, when they re-did their GDP estimates, they recently found out that their economy was almost double the size of what they previously thought and previously published. So that suddenly the Ghanaian economy was ranked as a middle-income country and was no longer eligible for concessional lending. Whereas other countries which have not updated their GDP statistics maybe are--we would hesitate to compare Ghana with Tanzania or Nigeria or Kenya today, and particularly it makes a mockery of those kind of rankings when we recently see, and how I describe in the book, how vulnerable these statistics are. But there are other, as you refer to as more sophisticated econometric analyses using these data. And I think perhaps the most famous debate for those who have been interested in African economic development for some time is that about whether structural adjustment programs or rather, the liberalization programs, which were implemented in sub-Saharan economics almost without exceptions from the 1980s, 1990s; and the big debate was whether this was supposed to spur growth, to make growth recover. And the big debates have always been to try to compare strong reformers to modest reformers, and then try to tease out an average GDP growth effect. Now, when we know how big the underlying availability of these data series are, we know there is enough error in there to make these kind of analyses completely--well, not trustworthy. Russ: I hear you soften it. Completely meaningless is what you meant to say. But 'not trustworthy' is very polite; I like that. Guest: Yeah. And the third example, which is one that I think is resurfacing as one of the, I think most important questions on how the poorest in the world are faring at the moment, and that is trying to get to what we refer to as elasticities between these measures. So, to what extent is recent growth causing a reduction in poverty, for instance. And when we look at papers written on that trying to calculate these in relationships between recent GDP growth and recent reductions or increases in poverty, these models are unfortunately way more sophisticated than the underlying data bases allow. Russ: And so basically you are trying to tease out the effect of liberalization on poverty. And you are saying this chain of causation--ideally, liberalization leads to higher growth, which should lead to less poverty. But your two data sets you are looking at, the two observations on GDP and on poverty, you really don't know what you are getting. Guest: That's right. And there is also--I think--in the book, I try to suggest one suggestion about talking about data as valid versus data as reliable. 'Valid,' talking about the GDP measure, would be the question whether the GDP estimate is correct. Does it capture the real economy 100%? Now we know that a GDP measure of the U.S. economy, the Germany economy, the Norwegian economy, will never be correct. It will always be a little bit off. Some data--there will be some cheating, there will be some data which are questionable. But we know we are more or less within bounds, off a couple of percentage here and there. And so that would be the question of validity. As we've seen from recent events in Ghana, and also forthcoming events in Nigeria, the validity question is really huge in sub-Saharan Africa. We are talking about plus-minus 50 to 100% on GDP levels. This would maybe not be a problem if you were interested in change, as we were talking about: what one type of change has a causal effect on another, such as GDP, liberalization, and parity. The problem is if you have that the validity of the measure changes through time. So that would be if you equated this with your bathroom scale at home--it wouldn't be such a big problem if your personal scale was off a pound or two, if you were basically just interested in measuring yourself on a weekly basis to see if you are gaining or losing. The problem that comes in is that of reliability, and that is if someone changes your scale in the middle of the night. And therefore you have a scale that shows an error in a different direction. And there you will have different problems talking about time series or changes over time. Another problem is that validity still remains with us even if the data was reliable, in that if you started comparing your own weight with that of the neighbor, who uses a different scale, then it would still be very different to determine who is the heaviest or lightest.

11:19 Russ: When you talked about Ghana I couldn't help thinking of my great grandmother, who at one point, her town was in Russia, and then because of some border changes I think related to WWI--could have been before that--her town was moved. She found herself now in Poland. And the joke was: Thank God, no more Russian winters! And I think about that. When you think about Ghana, they woke up one day and they found out they were twice as rich as they thought they were. And I guess in one dimension, that's a good thing. I suppose. And in another you'd say it really doesn't matter; it's like the Russian winter joke: If it snows a lot and it's cold, who cares what you call it. If you have a lot of money and you are living pretty well, who cares whether it's measured as rich or poor. But as you point out there are a lot of reasons that leaders of nations, particularly, and sometimes citizens care about how things are measured or classified. And in particular I would have thought coming into this book, coming into your book, I would have thought: Well, one of the problems of GDP measurement in every society, as you talk about in the book, is the informal sector, the underground sector, the non-market sector, the non-monetary home sector as it's called. There's all kinds of things going on that aren't measured, either because they are hidden or because they are simply not measured; there's no dollar value you can put onto them. And you give the classic example in the book of the person marrying his cook. You marry your cook; your cook is a woman, you are a man, you marry your cook. The cook's activity used to be part of GDP and now it's not because presumably she's not charging you for the meals she makes at home, at least in a monetary way. So, I understand that; and I thought, well, GDP in Africa is probably underestimated. But it's underestimated by sometimes an enormous amount. Which is what is so surprising. Explain why. You mention some of it a little bit but let's go into a little more depth. It's one thing, like what you say: Well, we didn't get it exactly right. But they are not even in the ballpark. And you suggest that sometimes it's incentives that are causing that. What's going on? Guest: Well, it's a nice little anecdote, you introduce this question, because one way of thinking about this is it is about boundaries or borders. And we are talking about what is referred to as the 'production boundary.' We decide what kind of economic activity should be included, accounted for, and which should not be included. In the book I provide a history of how African economies have been measured from colonial times until today. And one of the things that is particularly striking is how the views on production boundary does change through time. So, one of the reasons why one should--you could start thinking about maybe one should not include any non-monetary activities, or those which are not recorded. But that would be, as you say, it would render the GDP estimates completely meaningless because there is such a big part of the economy which takes part in this sector. And we know it's so economically important, so we need to include it. Another thing which is a good argument for why one should try to attempt an exhaustive measure is that one would get an artificial measure of growth as some economic activity travels from informal into formal activity, if you don't have a proper baseline estimate to begin with. But to answer that question about why there is so much economic activity outside of the production boundary, one of this is we have to look at the structure of African economies. And I made the point in the book that of course having inaccurate statistics is not solely an African problem relating to African economies. But I argue that there are reasons why we think this problem is particularly large in sub-Saharan Africa. One is that relating to the activity and the capacity of African states to collect information on its inhabitants. Usually these kind of collection of, having a population census, having a population registry, a registry of businesses, it has been related historically across the globe through the evolution through the formation of states and also across the evolution of taxation systems. In sub-Saharan Africa, historically, land has been more abundant; taxes have not been collected on private property to the extent it has been elsewhere. And that has meant that states have been more typically collecting information and taxes at borders. At ports. So that Frederick Cooper, for instance, talks about African states as gatekeeper states. This means that the states in Africa on average have less information about what goes on in their domestic economy, but also less incentive to collect that information. That's one type of more static structural characteristic. The other one which I point out as well is that African economies were subject to a deeper economic recession or at least a heavier economic shock in the 1980s. So when of course the petroleum crisis and the interest rates hikes which followed in the 1980s, which led to economic crisis in Latin America as well and stagnation in the world economy, was hitting African economies particularly hard. And it hit already weak states, particularly [?], so that budgets, which were allocating funds for statistical offices, were particularly constrained. Moreover, there was this double challenge in that states on average, before the 1980s on average were more engaged in organizing--they were directly involved in production; they were directly involved in transportation of goods; they were engaged in buying and selling of food products through their marketing boards. And basically, in the 1960s and the 1970s African states have more what we call 'administrative data'--data which they collected on their day-to-day basis. Post-1980, post structural adjustment, these states have increasingly been liberalized and therefore have access to less administrative data; and also have less incentive to collect this kind of information.

19:37 Russ: So basically one of the things that's going on is that the size of the unmeasured sector is changing radically over this time period. So you can't just sort of assume, as I think most people do about the United States--which would also be false, by the way--that, okay, we don't count household production. We don't count the meals that a husband or wife cook for the family. But that's okay because it's kind of constant. Of course, there have been radical changes in household production in the United States in the last 40 or 50 years. Year-to-year the changes are small. Decade-to-decade the changes are big even in the United States. Certainly in Africa they are enormous. The other part that I'd like you to talk about is the attempt to try to measure these unmeasurable sectors using proxies like rainfall or cement production. I found that tragicomic. It's unbelievable that that's what people are doing to try to estimate GDP. So, talk about that a little bit. Guest: Yeah. So let's sort of turn to what I started to describe at the beginning about how, why there are problems when you start aggregating GDP. So, if you are a national accountant and your task is to try to aggregate GDP, you find yourself having some problems reaching the total sum. For those who are readers who might not be that familiar with national accounting, so it might be worthwhile just going through the different ways of arriving at GDP. Russ: I would say very few of our listeners may be familiar with it. So, talk about that a little bit. I mean, in theory it's pretty straightforward: you just take the dollar value of everything people make and sell. It's straightforward. Goods and services; you just add them all up. Guest: That's right. So that's the theory behind it. When it actually goes on in practice it gets a bit more complicated. The system of national accounts, as it was designed by Richard Stone and later approved by the Organization for Economic Co-operation and Development (OECD) and different economies about how to do things--the United Nations have published three versions of the system of national accounts on how this should be done according to international standards. And the system is that you should reach a GDP in three ways independently. So, what you say: it's just about adding up what people produce or consume, and then get a sum more systematically. That is, your expenditure approach--which is the standard Keynesian identity--that GDP equals consumption plus investment plus government and then plus/minus exports and imports. So that should seem pretty straightforward. You just need to have the data on what the government does; you need to have some data of capital formation, investment; you have exports, you have imports. But then you come to consumption, which is the big unknown. Which in the absence of a fairly reliable household survey you simply do not know that share. So in practice consumption is always arrived at as a residual. So, this approach does not have that information in a statistical office in sub-Saharan Africa to arrive at this measure independently. The other approach, which used to, which is a little bit out of fashion, is deriving from political economy. That of adding together wages, profits, and rents. Which would be the old way of thinking--that GDP equals what everyone earns in the end. So it's just about the receipts of GDP. That would be the income approach. Because, again, large parts of the economy do not really receive a formal wage but are rather small scale independent operators who have a very unpredictable wage earning and pay themselves a wage when they can-- Russ: and barter. There's barter. Guest: Yeah. Different ways of getting by. Even if you have data on this, simply recall data. I think it would be difficult to compute. So this is also not, this approach to GDP is not calculated. What you do have is the production approach. Which means basically you go through the familiar industrial tables, where you have agriculture at the top; then you have mining; then you have manufacturing, and electricity, and water and construction. And then you reach different services such as retail and wholesale, transportation, hotels, restaurants; and then you arrive at the government. Their expenditures on different types of services, administration, educational, and health services. And then the big voluntary social services category where you would put Non-governmental Organizations (NGOs), churches, and so forth. That's at the end of the table. So anyway, that's the way that you, that GDP is arrived at--it is to try to go through this table and do the best you can. Per sector. Russ: So, where does it get tough? Guest: It does get tough already in the first category because in the agricultural sector we have very good data, quite good data, on how much agricultural goods are exported. Russ: And that's because they go out through a port where people are trying to measure them for government purposes. Of course, there's smuggling. But put that to the side. We have some idea of exports. Guest: That's right. And there has been an incentive since colonial times to collect these data, particularly when it was taxed. To take one example, in Uganda, which is a landlocked economy for those who do not know, behind Lake Victoria, behind Kenya and Tanzania, at the equator, in Central East Africa. Uganda collected their trade statistics in Mombasa, which is the port town in Kenya. So that would be a classic example of how the knowledge of the state was limited to particularly the port and the trade statistics. So, you have fairly good statistics there, but you don't have statistics on the food production, so there you will have to use a proxy. Typically, what statistical offices did in the 1960s when they started to aggregate these estimates was to rely on advice from the Food and Agricultural Organization in Rome, FAO, about what will be the reasonable per capita estimate for a country in their income bracket and then simply multiply that with their estimate of the rural population and population growth.

27:08 Russ: So that's kind of horrifying, because you are kind of assuming your answer. You are trying to figure out how much income people have, so you start by assuming what it is and then attributing some calorie consumption amount to that and then adding it to another thing you can't estimate very well, which is population. Guest: That's right. Russ: Bad. Guest: It is bad proxies; it is done in order to reach the total sum, to have the GDP aggregate. The problem becomes when people use these series in order to econometrically estimate other results. Russ: I'm reminded--sorry to tell another joke, but I'm reminded of the guy who, someone gives him $100 in $1-dollar bills. So he's counting to see whether he actually gave him 100 dollars: 53, 54,... 60,... 73, 74,... and he says: Well, if it's right this far it's probably right the rest of the way. It's kind of like: I estimated some of this stuff correctly; I'll just kind of get up to the total. It's bizarre. But let's--there are three main data series that econometricians use: The estimates of Angus Madison, the World Development Institute, and the Penn World Tables. I've seen all these data many times myself. It's not my area. But I was shocked to see them all in one place. Because what you usually see is someone will say: Well, I've used the Penn World Tables for my analysis; and you think: Well, that's a reliable source. Then you see that they don't agree so much. And we're not talking about being off by 1 or 2%. There are big differences. Guest: No, it's--when I did this exercise myself I was quite surprised, I must say. It came about to me as a thought experiment listening to some of these papers where someone found that some type of historical variable such as type of colonization, the engagement in slave trade, legal institutions after independence was causally related to GDP per capita. Today. Russ: I've seen this. Guest: Famous papers about how history matters for development. But that means you have to kind of trust the income distribution of African economies today. So you have to kind of think that those economics that are poor today or among the poorest are also among the poorest give or take a couple of years and also across data sets. And I asked someone--I sat in the audience at one of these papers and I said: Have you checked whether this is robust to using different data sets? And they said that they did. But I didn't trust them so I went home and did the exercise myself. Russ: Oh, you're very naughty. Shame on you. You skeptic! Guest: Absolutely. What I did--and this is part of my book, for readers who are interested--I did an earlier exercise like that; it was published in African Affairs in 2010 where basically I took the GDP per capita estimate in year 2000 for all sub-Saharan economies, according to the Madison data set, according to the World Development Indicator data set, and the Penn World Tables. And they use slightly different international prices, how they harmonize the data, different calculation behind it. But what I was interested in was simply how do they rank African economies? Difference in ranking. And what I found is that these data sets for the year 2000 agree on the ranking on one country and one country alone, and that is Democratic Republic of Congo. Which everyone agrees on is the poorest country in sub-Saharan Africa. Russ: We know something. It's good. Well, maybe. At least they agree. Guest: They agree upon that. It's also widely agreed that that's probably the GDP which is most poorly estimated in all of sub-Saharan Africa as well, showing that if you have little data you also tend to produce an underestimate. But then I looked through the rest--well, I ended up with 45 countries because not all of these data sets have the same countries. For example, Madison groups Ethiopia and Eritrea as one country, and so forth like that. So I ended up with 45 countries; and for the remaining 44, they did not agree on the ranking of any of these countries. And this was not the ranking being off with one or two spots. It was serious things such as the Penn World Tables thought that Liberia was the second-poorest country in sub-Saharan Africa; whereas Madison thought it was among the ten richest. Russ: Slight difference. Guest: And other countries such as Nigeria, Guinea, Mozambique, other countries traveled[?] a lot, and in fact the standard deviation in ranking if I recollect correctly was 7 ranks up and down. And my basic exercise from that was basically when we are looking at these rankings of GPD per capita we should approach this, well when in 2010 I thought 30% would be the correct reliability band. Today it's been showed by the recent GDP revision in Ghana we might be thinking plus or minus 50, 75% on this. And this means for all practical purposes all those econometric estimations are meaningless. Russ: Yeah. Totally meaningless. Guest: And that does not even--we should make this point: Most criticism of the use of the GDP estimates is about GDP is not a correct measure, we should be looking at other things, we should be looking at literacy and things like this. My critique is that there might be problems with the GDP estimate but how good is this particular data, is the subject of this book. And I show that there is something wrong with the metric itself, not only that it does not capture the right things.

34:04 Russ: So, one of the things--if you asked me to give a stylized fact about world development over the last 30 years, sort of what's the consensus, I would have said: The consensus is there's been tremendous growth over the last 20 years, particularly--a lot of it is in China and India, which is somewhat misleading because they are very large. Pull up the average. But the world overall--there are many fewer people living at less than a dollar a day, less than two dollars a day. And again you could broadly say this is due to liberalization, both China and India opened their countries to more market-based forces. So this informs my biases. I have to confess I like that. And then people say: The one continent, the one part of the world that hasn't participated in growth is Africa. Africa has been stagnant over the last 20 years. And then you say: Well, that's because they have bad institutions, they don't have rule of law, there's a lot of corruption, they don't trade much, there's a lot of protectionism. But what you are suggesting is that--I'm going to go a little further than maybe you want to; correct me. But what you are suggesting is that actually Africa might be doing pretty well. It's very hard to know. Because we have so much aid going to Africa, out of the total amount of aid--from the United States, from the World Bank, from the International Monetary Fund (IMF), from Europe--that it's in the interest of African leaders to look stagnant and to look poor. And maybe they aren't. Guest: I wouldn't go as far. Russ: I didn't think you would. Give me your reaction. Guest: I think I almost agreed with you until the last sentence. So I wouldn't go as far as--let's talk about those other things. Let's talk about the incentives for political leaders. At the very end. Because I think when you start looking upon the history--because what we are talking about here is really to reconcile time series data. So, how did African economies, looking at singular economies, on a regional basis, how have they performed over the last two decades? How did they perform over these past two decades, as compared to the three decades before that? That's the kind of debate we are thinking about. And in which time periods were they lagging behind Asia? In which periods were they lagging behind Latin America? When were they ahead, when did they perform better, and so forth? And I think there are a couple of striking things coming out of that. One is that, one example I would like to pull out and which relates to what we said about proxies earlier in the conversation; and I would like to tell a little bit about how GDP growth has been estimated in Tanzania. Russ: Yeah, that's a good one. Guest: Tanzania is an interesting country because it's been fairly well governed and so forth, but it's also special in that it was a country from independence, particularly from the late 1960s into the 1970s was also grouped under what we called 'African socialism.' So it was a socialist country in which the state was heavily involved in part of the economy, and also in trading, transporting goods, and so forth. And until the 1980s--actually, until the 1990s--the GDP series was made to assume that if the formal economy--that is, the recorded economy--was in decline, so was the informal economy. So that was an assumption of proportionality. That if food production as recorded went down, then we also think that food production not recorded went down. When the World Bank and technical advisers went in in the 1990s to re-base the GDP series, they made the opposite assumption. They then changed the assumption about how to use the proxies and then assumed that when the formal economy was in decline, then the informal or unrecorded economy would be growing. Russ: Could be true. Both those things could be true. Guest: Absolutely. Russ: We don't know which one it is. Or both over different periods of time. But it is--both those are imaginably true, that they either move in tandem or they move in opposite directions because of crowding out. Guest: Absolutely. And exactly what was the answer to that question was what attracted the most interest in development economics. This is related to how to interpret [?] surplus models, the classic Smithian models about what happens to the rest of the economy when there is an observable export growth, what happens when there is unlimited supply of labor, how must the other, unrecorded sector. So, so far these are big academic, scholarly questions, if you like. But what's very interesting about this case is that this question was settled at the statistical office. And it was settled differently before and after 1987. If you start looking at the resulting GDP series, what happened when they rebased the series was a sudden recovery of growth, when they changed assumptions. So that means that we must take care when we try to think about to what extent--so the question I put forward, in the case of Tanzania, for instance, the extent of economic decline was overestimated in the 1980s and the corollary would be that economic growth recovery in the 1990s following liberalization was overestimated. So that means, to the extent that we can generalize from the Tanzanian case--and I think one has to be careful about looking country per country basis here--we might be very careful about celebrating and applauding the recent growth we are seeing in sub-Saharan Africa. A second point which is interesting to make when it comes to the literature on African economic growth is not only have they ignored the problems of the GDP series but they have also been perhaps a bit too pessimistic, as you allude to. One interesting thing is--we know that The Economist had their famous special issue, their front page, calling Africa the 'Hopeless Continent.' That was in year 2000. The best seller perhaps on African economic growth would be The Bottom Billion by Paul Collier, published in 2007. Russ: Interviewed on this program. Guest: Which is a very interesting book. But it is perhaps surprising to look back at that and look at the GDP series. Which tells you that African economies have been growing very rapidly, indeed as quickly as other places in the world, since the mid-1990s. And somehow it took economists a decade and a half to notice it. Which is--and meanwhile I think what happened--that's a separate argument which I don't make in this book, which I made elsewhere, which I think to some extent the governance deficit, if you like, in sub-Saharan Africa has been overemphasized. We have been very good at--right now, economists and political scientists are very good at explaining why African economies are not growing. Well, in fact, they are. Which is an interesting knowledge problem. Russ: Yep. Fascinating. Guest: But there is also, relating again, how much should we trust these measures: I think a lot of the record high growth coming out such as in Ghana now and Nigeria to come, should be taken with a small bucket of salt because a lot of this sudden growth is coming by including previously unrecorded economy. Russ: Right. So you really have no idea. The bottom line is you don't have a benchmark that's reliable. Guest: That's right. Russ: And so as a result your rates of change are not reliable. Another way to say it is if your methodology is changing, you can't really compare the past, the future, and the present. Guest: Some of the basic caveats with interpreting African economic growth need to be taken as well, not only those related to statistics. I think the World Economic Outlook just published their projections and in there you see that--well, I can ask you. What do you think is projected to be the fastest growing economy next year? Russ: In the world? Guest: In the world. Russ: China! Guest: No. Southern Sudan. Russ: Of course. Why? Guest: Their GDP growth rate next year is projected to be 60%. Russ: It's a good year. Guest: Yeah. Then the next follow-up question is: Which do you think is the slowest growing economy this year in the world? Russ: Uh--Ireland. Greece. Guest: No. Southern Sudan, again. If I remember this correctly, their economy contracted by about 50% this year. Russ: Well, thank goodness it was only a one-year contraction. It's going to bounce right back. Guest: In Southern Sudan, this is about whether they are allowed to turn on the pipeline with oil or not. So here is where petroleum export growth matters more than statistics. One of the things that drives economic growth in sub-Saharan Africa is the undisputable rise in exports. How that relates to economic growth in the domestic economy and on the African continent is a question we are very poorly equipped to answer because of the measurement problem.

45:14 Russ: So, let me ask you: You are kind of preaching to the choir here, because as long-time listeners know I'm kind of skeptical about--I'm skeptical about econometric analysis for different reasons than we've talked about today. I'm skeptical because sometimes aggregation has methodological problems or there are too many things to measure that I think we don't measure. I think you quoted Einstein, what is it: Not everything that counts can be measured; not everything we can measure counts. So a lot of the time I think econometricians and economists doing statistical work--they take whatever they can get; throw it into a big data set and see what's correlated with everything else. And they find a story to tell. And there's a lot of ex-post story telling. But this is a whole new level. You are basically saying that we are really fooling ourselves. We've got a little time left; the obvious question is: What do we do? What do we do now? My end of the story, which Nassim Taleb tells, which I love, which is a bunch of people are lost, they are trying to drive--this is not the way he tells it but it's the same point--from Washington, D.C. to New York. They are struggling; they are off the main road. And finally one of them says: Good news; I just found a map. Oh, thank goodness, a map. Unfortunately it's a map of Paris. But somebody says: Well, that's okay. Some map is better than no map. And that's a common argument you hear. It's better to have some numbers. If you don't have any numbers, you can't say anything. But I would argue it's better to say something that's true than something that's unknowable true, or maybe false. Where do you think this leaves us? What do we know, if anything, that's reliable? And what might be done--you talk about this in the book--to make it better? Guest: I think that's a very fair question. And I think the answer depends on who I am talking to. One is those which are academics and scholars: how should they proceed when they conduct their next analyses. How can they be immune to these kind of measurement problems, if you like. Another thing is--so that would be the data users, of which there are expert users but there are also less expert users, such as headlines announcing this or that about economic growth in sub-Saharan Africa or elsewhere. But there is also that group of what-to-do-now questions should be addressed to the data disseminators. For instance then the World Bank, which is the big data bank. In publishing these data, what should they do in order to inform their users? And I think it's also important not to forget--my main reason for writing this is not only to inform data users, but also to map out a way in which this problem can be remedied for those economies we are talking about. So, actually a state's ability to have--the word 'statistics' refers to the state, having a valid GDP measure is basically a state knowing something about themselves. So I think there are some important questions regarding data production on the national level as well. Let me begin with data users. I think data users, I hope after they have read this book or heard this podcast, that they will not take a GDP per capita ranking of African economies seriously any more and start questioning. The fact of the matter is to compare GDP per capita as the official estimates you can download from the World Bank, from Ghana, Nigeria today, is completely meaningless. And the data will change for Nigeria next year, quite a bit. That's a direct implication there. The other is that if you run your regressions, if you are so inclined, then you have to take these kinds of error margins seriously and start thinking about using other physical data where you have export volumes and try to think about am I actually doing the right thing here. It's an interesting thing which I talk about in the book as well, that there has been a change in how data users inform themselves. I think development economists in the past tended to be much more writing books and monographs about countries, being an expert on an area rather than being a trend. Particularly in the 1990s, but still with us, although a lot of this is going into the randomized evaluations type of literature. There are still a lot more economists now relying on cross-country regressions than there used to be because of obvious reasons. Russ: Whoa, wait. What are those obvious reasons? Guest: The obvious reasons, one of them, is the availability of a computer. And the other obvious reason, the main reason, is the very availability of these data sets which I am discussing. So, when these things have been done laudable[?] on the finger tips, it's very easy to do research on the costs of economic development from your office. Which means that mistakes will be made. Which means that people do make mistakes when they analyze growth recovery in Tanzania because they don't know the changes in the underlying data series. So it means that people need to take one extra effort and actually do like historians do, and that means: What's the source of this evidence? What's the provenance? How good is this data? How did they come about? And so forth. Do some serious source criticism. Many economists do disseminating of data, as I talked about these are interlinked processes. But let's think about the World Development Indicators, for instance, where you can download all of these data or you look at the Millennium Development Goal reports where you have these big data sheets that are just filled out, where you have data on all the economies, and they seem to be one observation is comparable with the other. It is time for this, for World Bank data disseminators and others, to start labeling their product correctly. That means that there should be a double star on data that are actually not data but just projections. And there should be a product declaration which tells you that actually in Ghana they just revised their GDP series, so their level estimate is probably quite reliable, whereas that GDP estimates which we here publish side by side for Nigeria has not been revised since the year 1990 and is probably off by about 100%. So that means more meta-data, or information about the data.

53:14 Guest: When it comes to actually doing something about it, then we need to make a shift again to start realizing that this is about national statistical offices in different capitals and regions in sub-Saharan Africa. Russ: And you've spent time there, by the way. Guest: Yes. Russ: You do spend time in your office and you probably have a computer. You also talk in the book about--you ask people in these offices that are often run down and bedraggled and beat up and I think 'derelict' is a word you use--you ask them: What's going on here? And they often bravely do the best they can to defend their work. But often they confess to you that a lot of what they are doing is not very reliable. So you've gotten your hands dirty in some sense in the field. Guest: Yeah. I should--we didn't touch upon that. But I should point out that this book is the culminating result of effort done by conducting interviews and collecting information about GDP estimation in sub-Saharan Africa since 2007, and I have visited--in particular the research is based on visits to Accra, Ghana; Abuja, Nigeria; Kampala, Uganda; Nairobi in Kenya, Dar es Salaam in Tanzania, Lilongwe in Malawi; and also Lusaka, Zambia; and Gaborone in Botswana. Russ: I just have got to mention: Good luck in the Highlights there. We'll do the best we can to get those spelled correctly. [Um, Russ, international finance is my field and this whole podcast was right up my alley. I had no problem with this list and only had to look up if the 'es' in Dar es Salaam is capitalized and how to spell Lilongwe.--Econlib Ed.] Carry on. Guest: And we also conducted a survey by telephone and email interviews with the statistical officers. While my emphasis has been to be on the side of, to try to figure out what are the challenges being faced by the local statistical office. And I think that's also something that happens, and that's also part of the historical explanation. From the provisions of the United Nations statistical office and the legal provisions therein, this is data, the GDP estimates among other data was supposed to be submitted annually to the United Nations and then they would submit it and disseminate it through their bodies. So it was state data. At some point in the 1970s African economists among others were getting more and more delayed and were not able to provide the statistics because of the problems we talked about earlier in the show. And gradually the World Bank and the IMF started producing their own data. To the extent that when I give this paper and present my work, people are surprised that these GDP data are actually national statistical data. They wouldn't trust these data if they knew they were collected there. Somehow the brand name of the World Bank or the IMF is better. But so my book is an effort to try to think about, when you are quoting a GDP statistic on the Democratic Republic of Congo and Kinshasa, then you are actually making a statement about how reliable you think data coming from that office is. And that means we need to think about how we can strengthen those offices that are in need of support. Which is a considerable challenge. Which I write a lot about in my book, but maybe we don't have time to talk fully about here today.

56:51 Russ: We only have a few minutes, but I just want to add--it has crossed my mind at various times, because I am not very much any more an empirical economist. In my youth I ran regressions, like all good economists. I learned how to use the data package. But as I've gotten older I've gotten more interested in what I think of as narrative economics--trying to help people understand how the world works and less concerned about how to measure with any precision the connections between things. Because I think it's just to show people that there are connections. And people miss those anyway. Guest: Yeah. Russ: But, having said that--again, I'm not a consumer or producer of those data. But you know, in the United States--we are talking about Africa but in the United States--I view a lot of data that get produced by the government as being misleading or often "wrong." I think the data on inequality, which is household-based, is distorted by demographic change. The GDP numbers--they are probably in the ballpark. As you say. The errors are probably in the 1-3% range. They are probably similar, so it's probably not so bad. But it's funny--you talk about the asterisks. The Congressional Budget Office (CBO) has estimated that the stimulus package--at one point they estimated that the stimulus package of 2009 created anywhere between 500,000 and 3 million jobs. Now, at least they gave the range. Right? They didn't just give us the number. But they should have had an asterisk to say that this was not the actual number of jobs created the way the average human being would estimate it. They had run a set of regressions and basically done a set of projections based on the amount of money being spent by the stimulus package, and holding constant the impact of dollars of government employment that it held in the past. Which weren't the structural system that we were currently operating under. And how you can call that a measure of the effect of the stimulus is beyond me. Yet many, many, many mainstream economists will quote the CBO as showing that a nonpartisan Congressional Budget Office found the stimulus worked. Well, they didn't. They didn't do the analysis. And they confess it. They say so. They even say, when they report those estimates--they don't put an asterisk but they do have a paragraph if you find it that says: These don't look at actual data that occurred after the stimulus had passed but rather looks at the data that were before and then projects it forward as if that were the impact. And that's intellectually bankrupt, as far as I am concerned. So, you could argue--and I'll let you close on this--that we don't really want to improve these numbers. It's inherently always going to be politicized, misinterpreted. And even when you are trying to do the kind of analysis you are talking about, which is comparing, say, structural changes and their impact--some things are changing; you are probably going to end up doing something that is not so reliable anyway. So, tell me why we ought to be worried about this. Tell me why we ought to fix it. And I've been the cynic. So, you can close on a more uplifting note. Guest: Yeah. I'll do my best to transfer to that challenge. Russ: And you can bring in what you didn't answer before when I said that there was this incentive of leaders to distort the numbers downward because they want to keep collecting the money. So, you are going to have to answer that, too. Because that's part of the problem. If you want to have a better statistical office, in societies where trust and rule of law are not so established, they are going to be distorted anyway, even after you improve those offices. Guest: Hmm. I think it's important that we just clarify, suggest, that the political incentive to tamper with numbers. And I think what we've seen, for instance in the offered revision in Ghana and the forthcoming, impending offered revision in Nigeria, which do take these countries into middle-income poor country status, as was part of their political leaders' promises. Actually, in Ghana, they delivered on their promise 3 years ahead of schedule, which was quite something. And they will now, Nigeria as well, will probably become a bigger country in South Africa and therefore have more sway to negotiate for a seat in different U.N. or different legislative bodies. But I think that when it comes to--so there are some leaders, do want to show some evidence about doing better. And then some of them have increasingly turned away their heads to the statistical office, and say: Hey, maybe we are not estimating GDP correctly; maybe we can get a different type of estimate if we update the data series. Which is what will happen now in Nigeria. And Nigeria will become much richer than we thought it was. I think the relationship of the incentive there should not be interpreted that directly, but maybe the other way around, so that there are leaders who on purpose let their country look poor or more stagnant than it should be, and therefore don't up their statistics. I think those leaders who are not really interested in economic growth per say are not really interested in GDP statistics either. Russ: Fair enough. It's a good point. Guest: So, it just doesn't enter the calculus. Then, GDP I should emphasize as well is for me just an analytical angle and window in to looking at the statistical systems in these countries, and it is important because it is the sum of all the activities of the statistical office, and also the most widely used indicator for economic development. But underlying these data is all, what we talked about, different sectoral estimates, having a population census, being able to say whether inflation is at 10% or 120%--those kind of statistics are also needed in order to reach that final aggregate. And there are other places where this does apply. Particularly in food production. We know very well that governments have some incentives to overstate the type of food shortfall in order to qualify for financial aid. And also when financial aid is forthcoming, for instance has been discussed in the case of Malawi in the book, where Malawi for some years now has been overstating food production very, very seriously, in order to say: Hey, keep on giving us funds in order to subsidize maize, corn, fertilizers for maize production, because they are really, really working. To the extent that Malawi should at this moment--either the Malawians should be gaining weight quite rapidly or they should be exporting a lot of maize. But neither is happening. So the statistics must be wrong. But so those numbers do matter, and there are incentives to tamper with them each way. So it's important to be aware of this problem so it doesn't, as you talked about from the U.S. situation, treat these numbers as facts just because they are published by an official body. So that means, just not using the numbers that are convenient but try to take it, try to analyze what's behind it. That being said, I think there is a case for really working toward finding a strategy where we can get better numbers and better data for African economies. And that means, like I said, we have to shift our focus to realize that these are not data that are collected in Washington, D.C. They are actually collected in the countryside in Tanzania, in the countryside in Ghana. Or elsewhere. And that's where we have to think about what is needed.