The debate over whether aid works (and whether that is even the right question) shows no sign of coming to an end. Some people want you to believe that decades of foreign aid have essentially seen money flushed down the toilet. Others want you to believe that it money spent in Africa, Asia, Latin America and elsewhere is behind some major gains in the regions against poverty. Academics use research to make their case for and against aid.

Both sides have it wrong, says Yale University economist Nancy Qian. In a paper that has generated quite a bit of buzz among aid blogs, Qian looks closely at how aid is actually disbursed and the way it is analyzed. What she finds may not surprise many aid insiders, but the working is being heralded for saying what few are willing to say in a formal academic paper. The main findings in Making Progress on Foreign Aid are:

Aid is often times not about poverty alleviation. Studying the impact of aid by aggregating across donors, recipient countries, types of aid and time does not really help answer the question if aid works. Not enough data is available on how aid is delivered. The flow of aid and the top donors have been pretty much the same since 1960. Little aid money goes to the world’s poorest countries.

These findings then force a reshaping of the very question about whether aid works or not. She says that the current methods used to answer it harm the question that really matters: what can be done to make foreign aid effective. For example, with a significant portion of aid money being spent in donor countries, analyzing the impact of such expenditures on recipient countries is a bit more complicated. Grouping that spending with direct aid money muddy’s the waters.

“To ask whether existing aid has been effective is to ultimately ask whether foreign aid policy can be improved or should be re-designed,” concludes Qian. “In contrast, to ask whether foreign aid can ever be effective is to ask whether policymakers and researchers should devote any resources to understanding and improving foreign aid. For policymakers, it is critical to avoid using negative answers to the first question to answer questions regarding the potential value of foreign aid.”

Further, that all depends on how one defines some of the key terms, blogged Columbia University Political Scientist Chris Blattman. Not everyone agrees on what is and is not considered ‘aid.’ The Organization for Economic Development, one of the bodies that provides information on official development assistance given by the US, UK and other donor countries, has come under criticism lately. A fourteen-point reform plan was put forward by David Roodman for the Center for Global Development, which included no longer counting loans as development assistance.

Why does this matter to non-academics like you and I? It is important to know what is the best way for a country like the United States to support the growth of a country like Chad. Rigorous research can help contribute to answering that question. The problem is that current research does a poor job at identifying what different forms of aid do for different countries. That is because the definition of foreign aid differs greatly. Assistance to Israel and to Niger can be counted as foreign aid by the United States, despite having very different goals.

The analysis for aid needs to zoom in from 32,000 feet. Making such a shift will lead to better information and changed policies. It will help researchers better identify strengths and weaknesses in certain aid programs and initiatives.

If we are really serious about getting aid working (or, indeed, finding out what aid works, for what and in what circumstances), in addition to tackling our own political-economy of giving problems. We. Really. Need. To. Get. Better. Data. And do better testing. Data and methodology aren’t sexy and they don’t lend themselves to polemic. But if you want to know what works and why, they are essential,” blogged Terence Wood, a PhD student and researcher with the Development Policy Centre (Australia).

Right now, much of the research comes from pulling together data across a bunch of countries and donors. It is intuitive to say that all countries are not the same. Between 1.69% and 5.25% of foreign aid goes to poorest countries (bottom 20%) in the world. Making claims about the efficacy of aid and what can be done is rather difficult when so little money goes to the countries facing some of the greatest challenges.

Qian’s paper does not provide an end to the debate. Rather it points to a possible direction forward. The academics who are steeped in the subject are are paying attention.