A boy runs along a dirt road in Kogelo, western Kenya. (Photo: Peter Macdiarmid/Getty Images)

Last week’s New York Times Magazine featured a fawning Annie Lowrey profile of GiveDirectly, a charitable start-up that specializes in unconditional cash transfers for impoverished villagers in rural Kenya. As Lowrey discusses, GiveDirectly is currently undertaking a long-term project whereupon over 6,000 Kenyans in 40 villages will receive a “basic income” of around $22 a month for 12 years. GiveDirectly has been praised in recent years by charity watchers, in part because of its commitment to research-informed and evidence-driven philanthropy. And it’s now becoming a trendy charity in Silicon Valley, where universal basic income programs are a popular idea with programmers and engineers who can often see their work inching parts of our labor-force closer to obsolescence. But unmentioned in Lowrey’s profile is the fact that GiveDirectly’s sexy new basic income program seems to disregard evidence from its earlier work, namely that large and lump sum cash transfers seem to be more financially beneficial to recipients than the small, periodic payments of a basic income program.*

Jacob Kushner wrote for Pacific Standard last year about randomized controlled trials (RCTs), and their potential to significantly change the operation and influence of development aid. Randomized controlled trials involve the use of the scientific method to study the effectiveness of charitable development projects. Here’s Kushner:

RCTs use a control group that allows researchers to answer a question so rarely asked in the aid industry: Yes, this intervention seems to have worked — but might people’s situations have improved even without it? In science, it’s what’s known as the counterfactual.Then there’s the randomness factor. In an RCT, people suffering from poverty, poor health, or other ailments are randomly assigned to the control group or one of the treatment groups. The randomness solves the problem of individual choice — what if those who chose to participate in a given aid intervention were already the go-getters in their community, self-selecting in a way that would taint results?

Then there’s the randomness factor. In an RCT, people suffering from poverty, poor health, or other ailments are randomly assigned to the control group or one of the treatment groups. The randomness solves the problem of individual choice — what if those who chose to participate in a given aid intervention were already the go-getters in their community, self-selecting in a way that would taint results?

Though evidence-based study and analysis may seem like an obvious (and long overdue) technique — especially ina world of Spotify analytics and micro-statistics—Kushner argues it is a vital development for “a field known for justifying its existence using anecdotal, often emotionally charged success stories rather than data.”

In his article, Kushner extensively discusses GiveDirectly as a shining exemplar of a charity driven by the evidence from RCTs. And by most accounts, it is: Nearly all of its programs are done in coordination with an academic study, and the company is far more transparent about its results than most charities. So it’s surprising that, with its move to a monthly basic income program — and away from aid schemes with a focus on larger and lump sum transfers — GiveDirectly now seems to be flouting the results of its previous RCTs.

Previously, a Princeton University study of GiveDirectly’s earlier cash transfer programs found that large transfers of $1,000 performed significantly better than small, monthly transfers on most metrics. “We find large and highly significant differences between large and small transfers, all in the direction of ‘better’ outcomes for large transfers,” write the study’s authors, noting that the wealth increase resulting from the large transfer is “approximately twice as large as that for the small transfer.” Additionally, the lump sum recipients were more likely to purchase a metal roof, a durability upgrade resulting in an increase in long-term wealth. They also tended to have lower levels of stress hormone cortisol.

The positive results that GiveDirectly cites on its website are from the large $1,000 transfers, and its discussion of the $22 a month basic income program explicitly admits that there is not yet sufficient evidence supporting this method for smaller transfers. Additionally, according to GiveWell, GiveDirectly has estimated that its basic income program is 61 percent less cost effective (in terms of wealth increases per expenditure) than a lump sum cash transfer program. If one of GiveDirectly’s founding principles is RCT-based accountability, then why does its latest program seem to disregard past evidence?

One answer is that there have been few studies done of basic income programs, especially over long periods of time. GiveDirectly says this explicitly, writing on its website: “While the idea of a basic income has been gaining momentum recently, it remains controversial and has not yet been put to a sufficiently large-scale, long-term experimental test.” Although the short-term research of cash transfers that GiveDirectly has done so far has been more favorable for wealth increases toward lump sums, it’s possible that basic income programs could be more beneficial in the long-term by allowing people to make longer term plans with the knowledge that they will have a guaranteed income forthcoming. However, it is also possible that an inability to immediately invest the total eventual transfer amount will prevent the degree of wealth increase that a large, up-front lump sum would afford.

Another possible justification for GiveDirectly’s basic income program is that it may be a more likely candidate for a widespread governmental aid policy than a lump sum program. Indeed, this is the justification that GiveDirectly has given. According to GiveWell, GiveDirectly “expects the cost-effectiveness of direct transfers through this project to be lower than its standard program, it believes the potential for beneficial policy impact, which is hard to quantify, outweighs any difference.” And it’s true that a robust lump sum program would require significantly more initial funds to be immediately available upfront than a basic income program, which — although potentially more expensive in the long run — allows funds to be obtained over time, at roughly the rate of their distribution.

Additionally, GiveDirectly’s move to a basic income model might be a smart fundraising move. As mentioned above, universal basic income programs are popular with the tech world, and a charity that appears to be piloting such a program is likely to appeal to Silicon Valley philanthropists. Indeed, as Lowrey reports, a substantial portion of the $24 million that GiveDirectly has raised for its basic income program in Kenya has come from donors in the tech world, including “founders of Facebook, Instagram, eBay and a number of other Silicon Valley companies.” These donors, and, apparently, Lowrey, view GiveDirectly’s program not only as potentially important development tools, but also as an unprecedented test of a program that the United States might need soon enough, as Roombas replace maids and software replaces accountants. “Is Silicon Valley about to put the world out of work?” Lowrey writes, “And if so, do technologists owe the world a solution?”

However, it is unclear if GiveDirectly’s basic income program would be able to tell us much of anything about how a universal basic income would work in the U.S., should massive swaths of our population become unemployed due to automation. Out of work Americans would need significantly more than $22 a month to support themselves at even subsistence levels, so the program would be significantly more expensive. And the GiveDirectly basic income appears to be serving, for most Kenyans, as a supplementary income allowing them to better perform their jobs. Lowrey discusses one Kenyan who “used the cash to buy the first of three rounds of filament-thin fishing line that he would need to hand-knot into nets to catch tilapia in the lake.” This situation bears little resemblance to a post-automated U.S. filled with obsolete workers in need of total income support. Still, it is possible that GiveDirectly’s success in Kenya could help persuade philanthropists and lawmakers to explore universal basic income programs in more developed nations.

As Kushner writes, “RCTs are no cure-all; rather, they are the best tool we have to identify a whole range of cures that might, collectively, do the trick.” After the completion of GiveDirectly’s current 12-year program—which is itself randomized and controlled—we’re sure to have far more substantial evidence of whether long-term basic income programs in developing nations like Kenya might be a justifiable part of that collective trick.

*Update— March 3, 2017: This article has been updated to more accurately reflect the different outcomes resulting from large and lump sum cash transfers in the Princeton University study.