Evaluating foreign policy is hard

I’m posting this from Talloires, France, on the shore of Lake Annecy, at a lovely conference center run by Tufts University. The conference is on the Middle East (broadly defined to include Afghanistan), with a specific focus on European and American interests, policies, and perspectives on these issues. So far the panels and side discussions have been quite interesting but also pretty depressing, an atmosphere reinforced by lots of clouds and rain. The latter problem is actually a good thing, as France has been experiencing a punishing drought and needs the rain.

One issue that struck me during the discussions was the inherent difficulty of doing accurate "policy assessment," due in part to basic selection effects. To be specific, if you look just at the specific cases where some policy instrument (call it "Policy X") gets applied, it’s hard to know whether that policy is on balance a success or not. Policy X may only be chosen and implemented when dealing with really hard cases, for example, which are precisely the cases where it is least likely to work. (Among other things, the target state may have already considered the impact of Policy X and decided it can take the heat). But if adopting this policy in one case leads lots of other states to alter their behavior so that they don’t face similar actions, then these "dogs that don’t bark" are examples of the positive impact of Policy X that are unlikely to get factored into an assessment of its effectiveness.

A possible case in point is the attempt to alter Iran’s nuclear policy by imposing various economic sanctions on Tehran. If we look solely at Iran’s behavior, it’s clear that it hasn’t stopped nuclear enrichment and shows no sign of doing so. In short, the policy has failed. External sanctions have probably had a modest economic effect on Iran’s economic growth, but have clearly failed to achieve their main objective. But this failure is partly due to the fact that Iran was highly motivated to start with, which is in part why it rejected initial complaints about its nuclear program and eventually faced a series of escalating pressures. And it seems clear that Iran’s leadership decided ex ante that the sanctions would be bearable, or they would have backed down before they actually got applied. In short, sanctions got applied in precisely the circumstances where they were unlikely to work, and we shouldn’t be surprised that they failed.

But if a few other states were thinking a bit about acquiring nuclear weapons and took a look at Iran’s experience, and then concluded that pursuing the bomb just wasn’t worth all the aggravation, then Iran’s experience might have broader positive effects. It teaches that if you try to get a bomb, you’ll face censure, demands for inspections, lots of diplomatic hassle, and maybe even mildly inconvenient economic sanctions. Unless you really think you need a nuclear deterrent, who needs all these headaches? So while sanctions may have failed in dealing with a hard case like Iran, they may have helped reinforce global nonproliferation norms and thus persuaded a few other states not to start down that road themselves. And if that is indeed the case, then "Policy X" (in this case economic sanctions) may have a more positive "net effect" than a simple focus on Iran might suggest.

Of course, there’s a danger in this sort of reasoning: one can justify almost any confrontational policy by arguing that it is having far-reaching positive effects that extend well beyond a particular dispute. And that’s one reason that debates about foreign policy are so hard to resolve: it is too easy for all participants to make self-serving claims about the positive impact of the policy positions that they happen to prefer, and too hard to come up with a definitive answer. And dare I add that this is one of those places where academics can make a real contribution, by performing more sophisticated policy assessments that take base-line conditions and selection effects into account, so that we have a clearer assessment of the net effect of different policy tools.

All that said, I should probably add that I still think our confrontational approach toward Iran has been mistaken, insofar as it rests on inflated fears about Iranian capabilities and the implications of nuclear acquisition. But I’ll concede that making an example of Iran may–repeat, may–have helped discourage a few others from going down a similar route, and it would be interesting to know whether there’s any direct evidence of that effect on the calculations of other potential proliferators.