When a stupid op-ed produces some smart debate

So in yesterday’s New York Times, Northwestern University political science professor Jacqueline Stevens wrote something really stupid about whether the NSF should fund political science.

I don’t use the term "stupid" lightly. Based on her blog, she has a philosophy of science that’s about, oh, sixty years out of date. She was (as she now acknowledges) sloppy with some of her facts. One paragraph proudly trumps a John Lewis Gaddis essay that actually critiques the very kind of work Stevens claims to like. And, after spending much of the essay indicting political scientists for getting in bed with an imperial state ("research money that comes with ideological strings attached"), she closes with:

Government can — and should — assist political scientists, especially those who use history and theory to explain shifting political contexts, challenge our intuitions and help us see beyond daily newspaper headlines. Research aimed at political prediction is doomed to fail. At least if the idea is to predict more accurately than a dart-throwing chimp. To shield research from disciplinary biases of the moment, the government should finance scholars through a lottery: anyone with a political science Ph.D. and a defensible budget could apply for grants at different financing levels.

So, in other words, state funding is pernicious and corrupting — unless you and yours get the money.

So yeah, there’s a lot of stupidity contained in this essay. But that’s OK!! I have been to many a seminar (and maybe, just maybe, presented at some) in which the paper du jour was horrible, but the discussion that the paper triggered was quite interesting. And I think that happened in this case. For robust deconstructions of Stevens’ arguments, see Henry Farrell, Steve Saideman, Jim Johnson, and Jay Ulfelder.

Two other responses are worthy of note, however. At his blog, Phil Arena makes an interesting semi-serious suggestion:

Here’s a thought experiment — if [the American Political Science Association] were to increase membership dues by $500 a year or so, and if most current members remained members, we’d have a pool of money a bit smaller than the current NSF budget for political science, but still one that could fund a good number of projects with the greatest potential for generating positive externalities. The big data sets that lots of people use, like the NES, could continue. And let’s face it, many of the individual projects that are funded by the NSF do not generate significant positive externalities — and even if they did, a great many of them would be carried out even if without external funding. So the net loss wouldn’t be that big.

Now, there are some obvious problems and not-so-obvious problems with this proposal. Obvbiously, APSA membership wouldn’t stay the same size. Not-so-obviously, the demographics of APSA membership would likely skewresearch dollars in ways that people like Stevens would find even more abhorrent.

Still, I think a more modest version of this idea makes a great deal of sense. It’s entirely reasonable to, say, ask that tenured professors at R1 research universities to chip in $500 to a research fund. It’s also reasonable to ask other APSA members to chip in… something. I’d want to see the International Studies Association do the same. The result would not be a perfect substitute for NSF funding, but it would certainly be a good way of building up an appropriate research infrastructure free of Congressional interference.

Second, Penn political science professor Michael Horowitz posts about an ongoing research project with Official Blog Intellectual Crush Philip Tetlock. This section contains some beguiling findings… and an invitation:

One of the main things we are interested in determining is the situations in which experts provide knowledge-added value when it comes to making predictions about the world. Evidence from the first year of the project (year 2 started on Monday, June 18) suggests that, contrary to Stevens’ argument, experts might actually have something useful to say after all. For example, we have some initial evidence on a small number of questions from year 1 suggesting that experts are better at updating faster than educated members of the general public – they are better at determining the full implications of changes in events on the ground and updating their beliefs in response to those events. Over the course of the year, we will be exploring several topics of interest to the readers – and hopefully authors – of this blog. First, do experts potentially have advantages when it comes to making predictions that are based on process? In other words, does knowing when the next NATO Summit is occurring help you make a more accurate prediction about whether Macedonia will gain entry by 1 April 2013 (one of our open questions at the moment)? Alternatively, could it be that the advantage of experts is that they have a better understanding of world events when a question is asked, but then that advantage fades over time as the educated reader of the New York Times updates in response to world events? Second, when you inform experts of the predictions derived from prediction markets, the wisdom of groups, or teams of forecasters working together, are they able to use this information to yield more accurate predictions than the markets, the crowd, or teams, or do they make it worse? In theory, we would expect experts to be able to assimilate that information and use it to more accurately determine what will happen in the world. Or, maybe we would expect an expert to be able to recognize when the non-experts are wrong and outperform them. In reality, will this just demonstrate the experts are stubborn – but not in a good way? Finally, are there types of questions where experts are more or less able to make accurate predictions? Might experts outperform other methods when it comes to election forecasting in Venezuela or the fate of the Eurozone, but prove less capable when it comes to issues involving the use of military force? We hope to explore these and other issues over the course of the year and think this will raise many questions relevant for this blog. We will report back on how it is going. In the meantime, we need experts who are willing to participate. The workload will be light – promise. If you are interested in participating, expert or not, please contact me at horom (at) sas (dot) upenn (dot) edu and let’s see what you can do.

So, to sum up: a stupid op-ed. But lots of interesting things to read as a result of it. Well done, other political scientists!!