0:36 Intro. [Recording date: July 6, 2011.] You are critical of what might be called two extreme views of development and aid. One is associated with Jeffrey Sachs, who argues for large top-down approaches--from the outside--and one approach is associated with William Easterly, who argues that most aid is wasted and only a bottom-up, internal approach will work. What do you see as being incorrect with these two views? Not so much that I quarrel with the top down as much as that we know the answer to any problem and the answer is money. I think more often than not the problem isn't to be solved by just putting money in the most conventional way onto the problem. You have to think of why the problem is the way it is, try to understand it, try different solutions out, do the kind of experimental work we do at J-PAL--tryouts, strategies. And then you get to the answer. But that takes a while and it takes a lot of care. It's really not money or no money. It's money, but money with intelligence. You argue that the most effective, maybe the only effective way to get that intelligence is through a recent approach that has been taken, sometimes called RCT--randomized control trials. In layman's terms, running experiments with two different groups to try to measure the impact of a particular kind of intervention or form of aid. Talk about how those trials or experiments are structured and give us a flavor of how they are executed on the ground. Let me start by making a caveat. Our book is very careful to not quite say what you said. I think we are very careful to say that it is not just a matter of randomized control trials. It's a matter of presenting maybe the handy tools to answer these questions. But the book makes the point that it's much more fundamentally about knowing what the problem is and understanding why what you are doing is going to solve the problem and trying out different solutions. Sometimes the easiest and most effective way to try out the solutions and know the answers to those questions is what is called a randomized control trial, but that doesn't mean that it is by any means the only or even sometimes the most [?] of knowledge. Sometimes you can learn from many other facts that are staring at you. I think the main point we were making is broader, which is that it's very important to kind of understand that the on-the-ground mechanics of particular interventions. What is it that makes the particular problem we are trying to solve a problem? Why do we think that the intervention we are about to undertake is going to solve it? There are often glib assumptions about the causal sequence of the interventions to the solution, which we are liable to question and often find are inadequately thought through, the way policy is done. Establish exactly what we are trying to say. Why so much emphasis on these randomized control trials? What's nice about randomized control trials--two really nice things. First, they make it relatively easy to get a reliable answer to the question: Is this particular intervention working the way we are claiming it is working? The reason it makes it easy is it solves the perennial problem of all such evaluations, which is how do we know what would have happened if we hadn't done an intervention. Suppose you wanted to know the effect of providing computers in school. You choose some schools to have computers, and others not to have computers. Or the government so chooses. And then you find that at the schools that got computers, actually the students learned a lot more. Now, the challenge is to figure out: Does this mean it's because of the computers that they learned more, or is it because the government gave the computers to the schools where the students were more enthusiastic and were more interested, and that's why the kids learned more. You can easily conflate the reasons why the government gave the computers to those schools in the first place with the effect of the computer. This is a fundamental problem of all attempts to evaluate. This is the same problem when you look at the effects of microcredit. You look at the women who got microcredit and they are richer than the women who didn't get microcredit. But you don't know whether the microcredit is something people get on purpose--they go and find out about microcredit, join the group, get microcredit. And there's no reason to assume that the people who take the trouble to get microcredit are like the people who don't take the trouble to get microcredit, at least at this point of time in their life. Some of these people are enthusiastic about doing something. What randomized control does is it solves that problem of inference. It basically says: this school and that other school, or these hundred schools--their names are put into a hat and we draw out 50 of their names at random. And so the schools that did get the computers or the women who did get the microcredit are chosen at random, and that gives you the advantage that you can compare the two groups. There's no difference between them; decide by lottery which will be in each group. And therefore you can compare them without worrying about it. That's one big reason to do it. There's another reason, less emphasized but equally important. Suppose I wanted to know the effect of a particular computer program on learning. How would I go about finding that out? If I look in the world I might see that computer program used in some school. But it's not the case that it's just that program that's the only difference between those schools and other schools. Those schools might have different textbooks or different kinds of teachers. The blackboards might be different. You don't see one difference between schools; you see ten differences. Even if you just wanted to compare schools, and some schools have computers, you also get along with that the fact that those schools have many other different things. Very hard to unpack the effect of one particular thing, of a set of interventions into the constituent effects of individual interventions. One nice thing about doing an experiment is I literally choose: these 50 schools will be picked out a hat and I just give them this one extra thing. So, I can figure out what this one extra thing does. That way I can really, step by step, figure out what are the causal channels through which I can influence schooling.

10:05 Let me ask you a question about that. It's true that if you do it randomly, effectively as in a lottery, as if you are drawing names out of a hat, you have not biased the experiment yourself. But it does not follow that the 50 schools that get the computer program and the 50 schools that don't are effectively the same. They can have differences. So there is still going to be some statistical measurement of the program, I assume, to tease out the fact. You are exactly right. What a lottery guarantees is that these two groups are statistically identical. They are drawn from the same distribution--a word statisticians sometimes use for this--so they are identical in the same sense as if I toss a coin 50 times, and then I toss a coin another 50 times, those two groups of 50 tosses are identical. There is no systematic difference between them; and we can calculate exactly how much difference between them could arise out of random chance. So, it's not that we assume they are identical; but because we know how big these groups are and we know the laws of chance, we can figure out how much of the difference between these two groups can come out of pure random chance. Which is the point you are making. Therefore we can say that this difference is so big that it could not have come out of random chance, given the way we have drawn these things out of a hat. You are right--there is a role for statistics, and that role is to tell us how much of this difference could have come out of random chance. But we know exactly how to do that. Advantage of doing it is that we have a handle on the nature of that difference. Let me try to clarify that; I'm a little confused. If I try a particular software, let's say a learning tutorial. I make it available to students in 50 schools. And there are another 50 schools that don't get the tutorial. And then I try to measure after the fact its effectiveness. So, during the period when I decide which 50 schools, that's done randomly. But it could turn out that when I choose the 50 schools randomly, it still could be the case that the backgrounds, the socioeconomic backgrounds of the 50 schools are not literally identical to the 50 who don't get the tutorials. So, I would have to control for that, say parents' income or educational experience of the students that they might have that's different across the two samples. Don't I still have to control for those factors independently, to measure the independent effect of the tutorial? No. Actually, you don't. In fact, if you did that then some purists would say you actually vitiate the experiment. You are relying here on what is called the law of large numbers, which is: If I have 50 schools drawn out of a population of a hundred, then the probability that the 50 is systematically different from another 50, the probability that the first 50 is different from the average is something we can compute. What we are going to do is say that if the observed difference is bigger than the difference that could have happened out of random chance--we are going to let the data tell us the answer to that question, other than sort of trying to fix the random differences. One way to think about is imagine we had a million schools, and we drew half a million schools. By the law of large numbers, by the time you have a million all random differences will have canceled out. We know the probability exactly of how much--the probability that the mean performance of a group of half a million will be different from another group of half a million randomly drawn from the same population--we know exactly how unlikely that is. So, we can say with probability .99 these groups have exactly the same distribution of attributes, or probably .999 I know exactly how close these distributions are. I know how to control for the random differences between these groups rather than trying to actively control for it by putting in measures of those things. That would be a different way of doing things. I'm with you on a million. On a hundred, or 60, I understand it's still pretty high. But the bottom line is the methodology of how these are measured, you do not try to control for any differences that may have arisen. You assume they are the same population. I think you are saying something that I am slightly disagreeing with. We don't assume they are the same population. We assume we have randomized correctly and adjust our inference based on the fact that if I have drawn 50 out of 100, then the probability that they will [?]. When I report a result, any empirical study, I say with 95% probability the effect of this is greater than zero, or greater than 2 or greater than 5. That the claim that is always made. So, when I say that with 95% probability, that calculation of that 95% takes into account that these can happen by chance. In any study. So, when I make a statement that these computers made children better at math, I am making the statement with 95% probability made the children better at math. And when I say that, the calculation of what would constitute whether it's 95% or 93% or 97% will take into account the sample size--the number of computers, whether 50 or a million. I understand. I'm just wondering about the fact that there are these other underlying variables.

18:02 On to some of the findings. One of the fascinating things about the book that I really enjoyed. There's this famous dialog between Fitzgerald and Hemingway: I think Fitzgerald says, The rich are different from us; and Hemingway says, Yes, they have more money. One of the lessons of the book for me is that really isn't the only way to think about it, or a productive way to think about it. There are so many surprising behaviors and unexpected behaviors in these results and other findings that come from treating the poor with a little more care than they act like our models expect them to. I thought we'd start by talking about hunger. Some have argued that hunger traps the poor; if you are really hungry, you can't work very effectively, you can't be productive; if you can't be productive, you can't make much money, so you are going to be hungry. And so the poor are trapped in this cycle. What have we learned about how we react to the opportunity to get a little more food and how that affects their work productivity. I think it's very well established that when the poor get a little bit more money, they spend a bit of it to get more nutrition, but not a lot of it. They spend a bit of it to get more tasty food and a bit on other things. So, they don't act as if their one goal in life is to eat more, get more calories or protein into their bodies, and therefore become more productive. It's very clear that that is not what they are trying to achieve. As a result, if you look at the effect of extra money on extra productivity through this nutrition channel, I think people's view is this is relatively weak. Just by giving people a little more money we won't make them so much more productive that they could then pay that money back. It won't happen. A very optimistic idea would be you give these people more money; they go eat some more chickpeas and get stronger; then they make so much more money they can pay back the money you gave them to buy chickpeas. Mostly, they just don't buy chickpeas. They don't buy high-nutrition foods necessarily--chickpeas or soybeans, cheap high-nutrition foods. Why not? I think a lot of people assume that if you are near starvation, obviously you are going to channel that income into food because that's the biggest problem you have. But as you point out in the book, you tell that great story about the Moroccan person who was pretty hungry but he had a television. I think the poor recognize that even if they behave in the "most rational" way we posit for themselves, eat the chickpeas and all that, they are not going to get that much richer. Basically, this idea that I should postpone all pleasure for a little bit more money in a few weeks is just psychologically unrealistic. You are under a lot of stress, not a very pleasant life; you have this choice. If somebody came and offered you an opportunity to get really rich, said: If you dig this ditch for five days we'll give you a permanent government job with a good salary for the rest of your life--I think most of them will jump to that offer. That's not what is being offered. They are being offered a little bit more: maybe if I eat all the chickpeas I will get 10% more, and then if I keep eating those chickpeas every day I will get 10% more. Is that such a great deal? You are already poor and you have got this little extra money and you could have more fun for a couple of days; maybe that will keep you psychologically more able to deal with all the pressures you are under than having those chickpeas. It's not at all clear to me that if you took their psychology seriously that we would think that it always makes sense to go buy those chickpeas. You are really under pressure, haven't had much fun for a long time, worrying about all these problems all the time--may just want to do something relaxing for a little bit. I don't think that's any different from any of us--just have to make hard choices. You talk about how boring life can be in a small Moroccan village; the person you interviewed said television is better than food. Just a different way to get something out of life. You don't want to die; you are not starving to death. Reminds me of the issue in America where you give a homeless person money on the street and somebody says to you they are just going to waste it on alcohol; my view is if you are homeless on the street, sometimes a beer is a really great thing. Their life is hard; why would you deny them a little bit of pleasure because you think they should have a good meal that day? Just one meal, not life-changing. A little more nutrition or a little more thrill, and you might go for the thrill. Totally rational. Yes. I'm totally with you. I feel like if I were in that position, I don't know why I would be expected to make these kind of morally high-ground choices always.

25:09 Having said that, one of the interesting issues you discuss across a number of different examples that's so fascinating is opportunities as, say, farmers or parents where there is a technique available that could have what we would say a big bang for the buck. A huge rate of return. It might be using fertilizer for a farmer; for a mother it might be using hydration therapy to keep a child from dying from diarrhea or at least keeping them healthy. And yet often poor people in these countries don't avail themselves of these techniques. Talk about the challenges that the poor face in terms of information and trust, and how different their lives are. I think you kind of anticipated my answer partly. I think we kind of assume that when you tell someone this is what will make your child better--you take oral rehydration therapy and this will obviously make your child better--I think our presumption is that that sounds to them as it sounds to us. One core assumption there is they understand the process by which scientific knowledge is arrived at. We actually don't understand why oral rehydration therapy works much better than antibiotics than diarrhea. Maybe you do, but I don't. I'm not an expert in biology. I don't; that's okay, carry on. I think that's important. But the reason why we believe it is not because we know the science--I don't know the science--but because we know how the institutions which make such recommendations work, how they come up with these recommendations: They do RCTs on these recommendations and they find out this works better than that and then they come up with a recommendation. We know the process of knowledge generation; we have a faith. It's that we believe in a particular process of knowledge generation, we believe that process is more reliable than other processes. Now think of it from the point of view of poor people. I think the fundamental thing that is different is they don't understand the process of knowledge generation, because they never had the education. So they don't understand that there is a Food and Drug Administration (FDA) that does RCTs. So, for them this is just some knowledge that someone is claiming and that sounds counterintuitive. You mean water and sugar is better than a medicine injected into the body? Sounds insane. Medicines are good. Water and sugar is water and sugar. Ultimately oral rehydration is just water and sugar and salt. Why should that work better than the medicine? So, their reaction is: No, we shouldn't give them water because when they have diarrhea these kids are losing water all the time--which is why they die. If we don't give them water, they'll stop having watery bowels. So, they stop giving them water and go give them an antibiotic instead. Their logic--internally it's insane. They are making up a theory of the world from things they understand. In a sense they are less trusting of authority than we are. We can understand the process if a regulator at the FDA decides this or World Health Authority (WHA) recommends that. We kind of know what the WHA is. For the poor, I think they have often been told lies by the state; the government has told them things that haven't happened. So they are more skeptical of institutions generally. So, if you go and tell them things that are counterintuitive, like don't give them the medicine, give them this bottle of water with sugar and salt in it, that's one of the stories that these authorities make up; but God knows why they make those stories up. Or maybe they even think there is some evil plan there; and they should just go with their instinct. Medicine is better than water and sugar. If you state it like that, doesn't it sound plausible to you? Yes; not only that--a lot of times things we rely on and think of as totally scientific turn out to be not as scientific as we had thought. Skepticism is a good idea. The problem is we on the outside often see things we know have been tested, so consistently found to be effective that it's a "no-brainer." And yet children are dying because these techniques are not trusted. Part of that is an education problem; part is a worldview problem; part is as you say, unfamiliarity with the scientific method. And some of it is a distrust not just of the government but of strangers--people who come in from the outside saying, I know what's best for you. It's good to be skeptical of those folks. I agree with you. So, I think it makes perfect sense to react in this way. It's tragic but it's not stupid. It is based on some reasonable thinking.

31:21 Let's turn to the role of education, which is often thought to be the biggest single barrier to development; and as many have pointed out, there have been some dreadful results and attempts to improve education. This isn't only true in developing countries; it's true in developed countries as well. I always find it ironic that people look at the ineffectiveness of government spending on education via aid and forget how ineffective American spending on education often is in our own country. Let's talk about the big picture first. You contrast what you call the demand wallahs and the supply wallahs, where wallah is a term that means "provider of." Some people suggest we need to build more schools, create more teachers in these poor countries. Others say that unless there's a reason to go to school, a reason to invest, all the schools and teachers in the world won't matter. Talk about what's true and false about those views and what we do know that can make students better educated in poor countries. We know that the demand method--when it's clear that there are benefits from education, people put more effort into it. Also the supply method--when people have schools to go to, they learn more. It's not true that you can just tell people that you should want education and then magically schools will appear. There is a lot of clear evidence that school construction and more generally making schools more available does affect the educational level. Having said that, the reason why that debate is a bit besides the point is that if you look at where the big failures are, they seem to be inflated with our putting effort into sending these children to school and schools are there and still true that people aren't learning. That's the most striking thing. The striking fact is the lack of learning in settings where there doesn't seem to be any obvious lack of demand, or any lack of supply in the sense of there being a school, a teacher, etc. I think what makes it really interesting is that fact. You see a lot of private schools in developing countries. Interesting there are private schools that the poor send their children to, $1 a month or $2 a month kinds of private schools. They are all over the developing world. These parents are very poor and for them, $2 a month is a lot of money, so they are putting effort into this. They are making sure that their kids are getting something that they value. Yet even in those schools, you see slightly better results than the government schools, but still very disappointing results even in those cases. The average kid there isn't studying at great levels, either. The problem seems to be attitude of the entire education system--the teachers, the parents, and even the children toward what the goal of education is. They seem to have the idea that the goal of education is to get through some difficult exam and get some job having got through that. And that's something that only a few people can do it, what we call a winner-take-all education. It's like education is some long large gamble, something everybody should try, but it can't work out for everybody or for most people. Whereas in fact we know that most people get something out of being educated. Even if you can read a little bit you understand what these instructions are from the doctor; you do a little bit better in bringing up your children. The benefits seem to be much more widespread than people assume. The teachers seem to assume that most of these kids are hopeless; there's no point to trying to teach them anything. They just teach to the top of the class. And the parents don't complain, because they also think that the goal of the whole system is to train somebody, to find out whether your child is one of those few lucky geniuses who is going to go on to get a good education and a fine job; and if he isn't, what's the point of educating him? Everybody is much more pessimistic about the education outcomes of the median person than they should be. So, based on that, the curricula are way too hard, the teachers don't pay any attention to any of the children falling behind in class, the parents don't complain when the children fall behind in class--either they assume that there is some rough justice there, either your kid is really smart and the education is worthwhile, otherwise there is no point. So, everybody kind of colludes with that, and the kids very quickly lose hope. They kind of figure out that they are not one of these anointed people, and they start giving up. You see these children sitting through class after class where they understand nothing. They are in fourth grade, they can't really read. They are teaching history, they understand nothing of what's going on, but they sit calmly through school and start kind of dropping out. They vote with their feet. When they are in fourth grade they are too young; maybe keep coming. But by the time they reach sixth or seventh grade, they know the school thing is not working out for them, so they just drop out. You see this pattern over and over again, of unreasonable expectations that then are effectively used to clean out most of the people in the education.

38:27 You chronicle as many others have a huge problem with absenteeism by teachers, which is independent of this attitude problem you've mentioned. I'd like you to talk about the magnitude of the absenteeism problem, what might be done about that, and then what might be done about this attitude issue that the whole thing is a lottery for the elite. I think the magnitude is shocking sometimes. Roughly in many countries the actual teaching time that you get from teachers is about half of what they should be delivering, because a quarter of the time they don't show up and a quarter of the time when they show up they don't teach. If you take those two together, it's about maybe 50% of their regulation teaching time, they don't actually teach. I think that's a huge problem. A problem in some ways we know. What can be done about it? Some of my colleagues from the Poverty Action Lab, including Esther Duflo, my co-author on this book, did this experiment where they induce, actually had people teachers who would show up 60% of the time. This was in the old days before the information technology actually hit the camera. Still in the days when you had film cameras, cheap film cameras given to every teacher; and the teacher was told: at the beginning of the day, have a child take a picture of you with at least 8 other children--minimum class size--and at the end of the day do that again. And the camera had a date and time. The arrangement with the teachers was that they would be paid a fixed amount plus a pro-rata amount based on the number of days they attended, and this date and time stamps were just to give proof that they had attended. So, this was proof; based on having the proof, they would get paid more for every day they attended. You look at what that did--it halved absence rate and the children learned much more. Channel there of how to improve education. It is something straightforward to do if there is political will to do it. I don't think this is a difficult problem, as such. These days with fingerprint, you could automatize the system with very little cost and make it much more effective than this. This was a system that was designed for the world ten years ago; now you could do it much more effectively using information technology. So, that's the easy part. Then there is a hard part, which is how do you get teachers to have different attitudes. That is not easy but not actually as hard as it seems. One thing that is kind of heartening is that when the same teachers, the ones who teach in government schools and ignore most of the children and act in this elitist way--when you train them to teach in a summer school for the lagging children, the lagging children actually learn a hell of a lot from them. The teachers are very effective in teaching the lagging children. It's striking that the same person who actually ignores the lagging children when he teaches in regular school seems to be very effective in teaching those children when in summer school that's his job. So, it's a matter of the education system sending the signal that this is what we want. It's not a huge battle to be fought. Some teachers have obviously checked out; but they are willing to try to teach. It's just difficult for them to try to teach these lagging children in the regular school system because the whole school system expects them not to do that. Interesting challenge. It's a challenge in America, too, where there is I think a surprising lack of oversight in the public school system.

44:11 One of the things it reminds me of, that I think was so interesting in the book, is how many poor people in poor countries aspire toward government jobs, and often as teachers. This is a little disheartening, although when you think about the opportunity for absenteeism and drinking tea, which is one of the things you mention these teachers often do--even when they are in school they often are hanging out drinking tea--you can understand why a government job is so attractive. And often the private sector in some of these countries is not very well organized. For a variety of reasons, governance being one of them, it's hard for the private sector to generate employment through legitimate channels. Talk a little bit about what hope we have in that area. I think that in general the problems of the private sector in developing countries is interesting. You see huge numbers of private sector stores--you see how many small stores there are on the street, how much private enterprise there is in these countries. Also true that if you look at the productivity of the private sector, it seems very low. I think that sort of poses an interesting challenge: Why are all these people in the private sector if the private sector is not very productive? Turns out, not surprisingly, the reason is the private sector is not productive. Because there are no good jobs around, people substitute by setting up their own little firms and trying to eke out a living by selling what they can sell or doing some service that somebody may want to buy. So you end up with a huge number of inefficient firms. What all these countries need to work is to have much bigger, more productive and less numerous firms. That's the big challenge. We've been told this is going to happen from microcredit; but I think microcredit is the wrong instrument to generate large firms. Second, loans have the structure where you are required to pay back the loan in the weeks after you got the loan. You pay back a chunk of money every week, so you are really stuck. You have to get your loan to generate an income immediately. That doesn't make for being really an entrepreneur in the adventurous sense of the word. You just find the easiest, simplest, most reliable thing you can do which you can start tomorrow and generate some money immediately and you do that. Very unlikely that you are going to become big and productive.