MalAdaptive: The Political Economy of RCTs By Bryan Caplan

The gold standard of modern social science is the bona fide experiment, also known as a “random controlled trial.” “The stuff going on at the Poverty Action Lab” is the modal answer to the standard GMU lunch question, “What’s the most important work going on in social science today?” And yet… real-world policy-makers continue to neglect, evade, and actively oppose experimental tests of efficacy. In Adapt, Undercover Economist Tim Harford explains why:

[P]oliticians resist pilot schemes with objective measures of success. This is partly because politicians are in a hurry: they expect to hold on to a role for two or four years, not long enough for most experiments to deliver meaningful results.* Even more politically inconvenient is the fact that half of the pilot schemes will fail… so the pilot will simply produce stark evidence of that failure.

Harford continues:

This is our fault as much as the fault of our politicians. We should tolerate, even celebrate, any politicians who test their ideas robustly enough to prove that some of them don’t work. But, of course, we do not.

This is all a nice example of a theme I’ve been pushing for a while: Most political “agency failures” actually reflect principal negligence:

[P]olitical agency problems are often a byproduct of voter irrationality. The principals give their agents grossly suboptimal incentives, then complain that the agents fail to carry out their assignments. For example, a key feature of the main models in PA is that there is no pay-for-performance. No matter how good or bad a job a politician does, he gets the same compensation. Admittedly, this is a standard feature of modern democracies. But why is it a standard feature? Because it is too hard to evaluate politicians’ job performance? If so, using re-election as a carrot is equally misguided. Because it is too hard to assign optimal weights to various aspects of job performance? If so, one could simply “let the people decide” the optimal weights by basing bonuses on approval ratings. Because politicians’ actions have long-run consequences? If so, bonuses could be a function of long-run consequences… Some better arguments against paying politicians for performance may exist (Hart et al. 1997). However, the flimsiness of the leading objections should open us up to a simple alternative: Pay-for-performance is a good idea, but the public is too irrational to accept it.

The lead article in the latest Journal of Political Economy presents a model where voter turnout automatically adjusts to overcome ideological bias. What’s next: A model where it’s rational for voters to punish politicians for using RCTs?



* Harford’s own footnote: “Donald Green, Professor of Political Science at Yale, tells me that one question in the social sciences has been thoroughly tested with field experiments: how to get out the vote. So politicians can use rigorous evaluation methods when it suits them.”