



Simple cost-effectiveness measures

For LLINs, we estimate a cost of $1.36 per “person protected by an LLIN per year.” (This is based on $5.15 per LLIN distributed, with each LLIN covering an average of 1.8 people for an average of 2.22 years; details at the spreadsheet linked from our cost-effectiveness analysis of LLINs, column H).

(This is based on $5.15 per LLIN distributed, with each LLIN covering an average of 1.8 people for an average of 2.22 years; details at the spreadsheet linked from our cost-effectiveness analysis of LLINs, column H). For deworming, we estimate a cost of $0.51 per person treated (details). This $0.51 figure incorporates costs that SCI, our top-rated deworming charity, reported to us. We have not fully investigated all costs associated with deworming programs (as we have for bednets and cash transfers) and believe the cost per person treated could be ~10-25% higher, leading to a cost per person treated of $.51-.64.In high-prevalence areas, deworming is annual, and all of the information we have about the impacts of deworming is based on high-prevalence areas. There is an additional question of what percentage of people treated are children, since the main case for deworming (in our view) is its developmental impacts when applied to children; we have little ability to estimate this figure and generally assume that 50% of those treated are children, leading to a figure of $1.02-$1.28 per child treated .

(details). This $0.51 figure incorporates costs that SCI, our top-rated deworming charity, reported to us. We have not fully investigated all costs associated with deworming programs (as we have for bednets and cash transfers) and believe the cost per person treated could be ~10-25% higher, leading to a cost per person treated of $.51-.64.In high-prevalence areas, deworming is annual, and all of the information we have about the impacts of deworming is based on high-prevalence areas. There is an additional question of what percentage of people treated are children, since the main case for deworming (in our view) is its developmental impacts when applied to children; we have little ability to estimate this figure and generally assume that 50% of those treated are children, leading to a figure of . Cash transfers are much more difficult to model on a “per person per year” basis, since they are structured as one-time “wealth transfers.” One way of coming up with a partially informative “per person per year” figure is to look at the cost of a metal roof, which is a commonly reported use of cash transfers. GiveDirectly estimates that such a roof costs about $250 for a household and lasts about 20 years. During our visit to GiveDirectly’s operations in Kenya, recipients reported spending closer to $500 for a roof. At an average household size of 4.7 (from the “Household size analysis” document in our GiveDirectly review) and 20 years per roof, that implies about $2.66-$5.32 in recipient expenditures per “person-year of roof coverage”; at a rate of 90c transferred for each $1 in expenses, that implies about $3-$6 in donor expenses per “person-year of roof coverage.”

The obvious problem with these figures is that we don’t know how beneficial a year of LLIN coverage is relative to a year of deworming or a year of coverage by a roof. However, estimating such benefits involves a lot more judgment calls than estimating the costs. We think the figures listed above – while inconclusive and unsatisfying – are much less likely to be wildly inaccurate on their own terms than the figures that follow, so we think it’s appropriate to keep them in mind.

Humanitarian outcomes

The cost per life saved for LLIN distribution

Our current best estimate of the “cost per life saved” for LLIN distribution is about $2300 (details at the spreadsheet linked from our cost-effectiveness analysis of LLINs, column H). Some notes on this figure:

This figure is only including direct lives saved for children under five . It is also possible that LLINs save adult lives, but figures on adult malaria deaths are disputed and we have not seen studies addressing whether LLINs save adult lives. As discussed previously, we believe the case that LLINs have non-mortality-related “developmental benefits” is somewhat comparable to the case that deworming has such benefits. Finally, LLIN distribution may reduce the burden of malaria, on LLIN users and on the health system, in ways not captured in the above considerations.

. Our spreadsheet also estimates the cost per life saved under different assumptions about questions like “How long do LLINs last in the field?” and its overall range comes out at ~$1700-$5500.

There are several potential major sources of uncertainty in our estimate that are not captured in our spreadsheet, including the possibility of insecticide resistance, the possibility that today’s conditions differ in other ways from the conditions under which the original studies of mortality effects were done, and more. These are listed in our discussion of LLIN cost-effectiveness.

The cost per “equivalent life saved” for deworming

We see a lot of uncertainty around the benefits of mass deworming. We have not seen evidence that it directly saves lives, and any such lives saved are likely to be quite rare (not competitive with how often LLINs save lives). However, there is some evidence suggesting that deworming has developmental impacts: that deworming someone in childhood can cause them to earn more money later in life (among other benefits).

When attempting to estimate a “cost per life saved equivalent” for deworming, one must take a view on multiple difficult-to-quantify factors, all of which are listed in the “assumptions” sheet of our spreadsheet on “cost per life saved equivalent” comparisons:

The relative value of saving a life vs. realizing the developmental benefits associated with deworming. The value assigned by the corrected Disease Control Priorities Report to averting “cognitive impairment” is 2.4% of the value it would assign to averting death, but many might consider improving a life to be more than 2.4% as valuable as saving a life. (Assumption 9 on “Assumptions” sheet)

The proportion of people dewormed by SCI who are children. (Assumption 2)

Whether the benefits of deworming are likely to scale linearly with repeated treatments. The Kenya study on developmental benefitslooks at the benefits of receiving ~2.5 years of additional treatment; it’s possible that people treated by SCI receive fewer or more years of treatment. (SCI generally aims for repeated treatment throughout childhood.) (See Assumption 3; we formalized this issue via a multiplier that captures “how helpful years of treatment in SCI’s program are, relative to the years of treatment in the key study.”)

Whether to make a “replicability adjustment” due to the fact that developmental benefits for deworming are not as robustly established as mortality effects for LLINs. (We have discussed this issue previously.) We have sought a reference point for how likely it is that a given study result will hold up under replication, and have seen some potentially relevant figures in John Ioannidis’s analysis of biomedical literature (here and here), though the analogy between biomedical studies and the studies in question has substantial limitations. We have formalized this adjustment as Assumption 4 in the sheet.

Whether to make an “external validity” adjustment, to account for the fact that infection prevalence rates were unusually high due to El Nino in the Kenya study on developmental benefits (the study that provides the most helpful information for quantifying the benefits of deworming). We have formalized this adjustment as Assumption 1 in the sheet.

How to account for the shorter-term health benefits of deworming, including both rare severe health effects (such as intestinal obstruction due to ascariasis) and subtle general health effects. Given recent developments, we think it is reasonable to consider the subtle general health benefits negligible, but we also provide the option to use our previous estimate of such benefits, which is based on the Disease Control Priorities Report.

Our spreadsheet on “cost per life saved equivalent” comparisons shows the “cost per life saved equivalent” under different assumptions. We have provided many possible assumptions in the “assumptions” sheet and calculated the “cost per life saved equivalent” for deworming under each possible combination of inputs; it ranges from $31 in the most optimistic scenario to over $2 million in the most pessimistic. It also includes scenarios that represent different GiveWell staff members’ best guesses, discussed further below.

Comparing LLIN distribution and deworming

Recall that LLIN distribution may also have developmental benefits. Generally, the more optimistic one is about the case that health in childhood affects quality of life in adulthood, the more optimistic one ought to be about both LLIN distribution and deworming. Personally, I would guess that LLIN distribution has stronger developmental benefits than deworming on a “per-person-per-year” basis (for reasons outlined previously) though one could easily argue either side of this question.

If one assumes that LLIN distribution and deworming have equal benefits on a “per-person-per-year” basis, and that the proportion of people treated who are children is similar for the two, then this implies that when considering developmental benefits alone, deworming accomplishes about 2.5x as much good per dollar as LLIN distribution (based simply on the “cost per person treated per year” from the previous section).

One may also try to combine the mortality benefits and the developmental benefits of LLIN distribution under these assumptions, as we do in our spreadsheet on “cost per life saved equivalent” comparisons. Below, three of us – myself, Elie, and Alexander – explain our assumptions and give the resulting cost-effectiveness comparison. We also encourage readers to use the spreadsheet to enter their own assumptions. You can do so by going to the “Master” sheet in our spreadsheet on “cost per life saved equivalent” comparisons, manually editing columns M through V for any row, and watching the output in columns X and Y.

Elie’s assumptions: $0.56 per person dewormed (assumes that our $0.51 estimate is too low by 10%, based on an underestimate we previously made of AMF’s costs before analyzing more deeply); 50% of deworming goes to children; child-years of deworming by SCI are on average 1/3 as effective as child-years in the key study; developmental benefits should be valued 20% as much as lives saved; the external validity of the key deworming study is 30.25% (based on the change in moderate-heavy infections experienced during the course of the study); 40% chance that the study would hold up under replication; standard (Disease Control Priorities Report-based) minor health benefits of deworming. $1,668 per life saved equivalent for deworming (83% from developmental benefits); $1,564 for nets.

Holden’s assumptions: $0.51 per person dewormed (uses our current estimate); 50% of deworming goes to children; child-years of deworming by SCI are on average (2.41/10) as effective as child-years in the key study (this is equivalent to assuming that SCI is treating all children throughout childhood, and that benefits do not increase beyond the 2.41 years point); the external validity of the key deworming study is 30.25%; 10 people with developmental benefits are morally equivalent to 1 life saved (this figure comes from regressing my own intuitions about what’s valuable – which put very low value on saving lives relative to improving them – somewhat toward “normality,” since I’m not confident that my intuitions are appropriate on this point); 30% chance that the study would hold up under replication; standard (Disease Control Priorities Report) based minor health benefits of deworming. $3,813 per life saved equivalent for deworming (56% from developmental benefits); $2,004 per life saved equivalent for nets.

Alexander’s assumptions: $0.51 per person dewormed (uses our current estimate); 50% of deworming goes to children; child-years of deworming by SCI are on average exactly as effective as child-years in the key study (this is based on ignorance about where in the distribution of child-years of deworming additional funds might be spent, and takes into account the upside potential of later years conceivably helping to eliminate schistosomiasis from an area); the external validity of the key deworming study is 30.25%; a 2.4% quasi-disability weight for developmental effects (following the Disease Control Priorities Report, not due to confidence that it is the correct estimate but because it roughly maps to the intuition that a permanent 25% increase in income [derived from Baird et al 2012] for ~40 people starting in adulthood would be roughly as valuable as saving a life of a young person); 50% chance that study would hold up under replication; standard (Disease Control Priorities Report 3% discount rate DALY-based) minor health benefits of deworming (with no minor health benefits for bednets). $2,981 per life saved equivalent for deworming (73% from developmental benefits); $2,027 per life saved equivalent for nets. (Alexander comes up with broadly similar figures because he’s more confident in the representativeness and quality of the deworming studies than Elie or I, but places much less weight on developmental effects than either of us.)

All of these figures can be converted straightforwardly to DALYs if one prefers these units. The DALYs per “life saved equivalent,” for three different versions of DALYs, are given in column AC of our spreadsheet on “cost per life saved equivalent” comparisons.

Cost per “equivalent life saved” for cash?

We have not attempted to use the “equivalent life saved” framework to quantify the benefits of cash transfers, since we know so little about how they are spent and what the likely results are. Instead, we compare cash transfers to the other interventions using a different framework, discussed immediately below.

Financial returns

We can estimate a similar figure for cash transfers, using what we know about the longer-term returns to cash transfers, and compare the two. Finally, we can do a similar calculation for LLIN distribution, assuming that LLINs have similar developmental benefits to deworming (on a per-person-per-year basis), though this calculation excludes other benefits of LLINs such as direct mortality benefits.

Note: we have updated this section (and the linked spreadsheets) since initially posting to incorporate the short-term health benefits of deworming; across all scenarios we assume that each dewormed person experiences current health benefits of value equivalent to a 0.51% increase in income. This value was chosen because it results in 71% of the total benefits of deworming in our preferred scenarios (and 60% in the full set of considered scenarios) arising from developmental effects, the same as in the “lives saved equivalent” framework discussed above (assuming the short term health effects of deworming from the DCP2).

Our spreadsheet on financial returns performs this comparison. As in the previous section, many assumptions and judgment calls must be made. Regarding deworming, we have the same questions as in the previous section regarding the reliability and external validity of the studies on developmental benefits. Regarding cash transfers, there are additional questions: how much of the transfers are likely to be invested, and what rate of return are they likely to earn? What discount rate should be used to capture the fact that (a) recipients prefer present consumption to future consumption; (b) donors can invest their money and give later rather than giving today and letting recipients invest it?

Our spreadsheet on financial returns (XLS) addresses these questions.

Again, because this calculation is extremely sensitive to small changes in inputs, we provide the inputs for myself, Elie and Alexander. We also encourage readers to use the spreadsheet to enter their own assumptions. You can do so by going to the “Overview” sheet in our spreadsheet on financial returns, manually editing columns J through Q for any row, and watching the output in columns S through Z.

Elie’s assumptions: 50% of deworming goes to children; child-years of deworming by SCI are on average 1/3 as effective as child-years in the key study; 30.25% adjustment for external validity concerns; cash transfers are 75% invested, returning 10% a year with a 5% discount rate; 40% replicability adjustment for deworming and a 95% replicability adjustment for cash. (Elie assigns a much lower likely rate of return than seen in the studies on cash transfers, and as such does not do another large “replicability adjustment” for this.) This results in deworming being 2.3x as cost-effective as cash, with each dollar spent leading to total benefits equal to 1.25% of annual income for deworming, vs. 0.55% for cash. (In a family with the sort of income reported for GiveDirectly’s clients, this would translate to the equivalent of $2.98 in benefits for every dollar donated for deworming; $1.30 for cash.)

Holden’s assumptions: I did two different estimates, one representing low skepticism about results found in studies and another representing high skepticism. In both, I assume 50% of deworming goes to children; that child-years of deworming by SCI are on average 50% as effective as child-years in the key study; a 30.25% adjustment for external validity concerns (based on before-and-after rates of heavy-to-moderate worm infections in the key study); and a 75% investment rate (i.e., cash transfers are 75% invested vs. 25% consumed) with a 5% discount rate. However, in one case I give both the cash and deworming studies a 50% chance of holding up under replication, and use a rate of return (25%) that is triangulated from studies; in the other case I use a 5% rate of return on investment (which effectively assumes simply keeping up with the discount rate, i.e., not earning any substantive return on investment) while applying a 30% “replicability adjustment” to the deworming study. These assumptions result in deworming being ~2.9-4.2x as cost-effective as cash. In the less skeptical version, each dollar spent on deworming leads to total benefits equal to 1.93% of annual income, as opposed to 0.66% for cash; in the more skeptical version, each dollar spent on deworming leads to total benefits equal to 1.36% of annual income, as opposed to 0.32% for cash. (In a family with the sort of income reported for GiveDirectly’s clients, this would translate to the equivalent in $4.58 (optimistic)/$3.22 (skeptical) in benefits for every dollar donated for deworming; $1.57 (optimistic)/$0.76 (skeptical) for cash.)

Alexander’s assumptions are similar to those in my “optimistic” scenario except that he (as described above) assumes that 100% of the child-years of deworming are as effective as the child-years from the key study and takes the average of the reported ranges for cash transfer ROI across the two longer-term studies (54%). These go in opposite directions, leading to the conclusion that each dollar spent on deworming translates to total benefits worth 3.36% of annual income ($7.97 in a family with the sort of income reported for GiveDirectly’s clients) while each dollar spent on cash transfers translates to total benefits worth 1.21% of annual income ($2.87), which implies that deworming is ~2.8x as cost-effective as cash.

Relative to the ranges of assumptions we consider, our individual estimates come out more positive for cash transfers. The mean across the 6,075 scenarios representing each combination of our assumption set is that each dollar spent on deworming increases annual consumption by 3.37% and each dollar spent on cash transfers increases annual consumption by 0.65%, implying that deworming is ~5x as cost-effective as cash transfers.

Conclusion