In a recent think piece “Drone blowback in Pakistan is a myth. Here’s why”, political scientist Aqil Shah argues that blowback from American drone bombardment is a myth. He recently penned this article for The Monkey Cage, a political science blog at the Washington Post, run by academics but meant for the general public.

Shah, an assistant professor at University of Oklahoma, tells us that he conducted 147 semi-structured interviews with adults from North Waziristan and found that many of his respondents support drone attacks. Based on this, he argues that the contention that “drones strikes inadvertently increase terrorism by exerting a 'blowback effect' is unsupported."

Not only do we disagree with the underlying assumptions of Shah’s arguments, but we find that the article even fails on its own terms. We prove below how he sets up straw men by mischaracterizing the arguments of his opponents and incorrectly implies that his study is representative. And even if one accepts these flaws, the results still don’t support his conclusions. Irrespective of one’s views on drone attacks, this article is methodologically flawed and misleading. The assumption that underpins his research – the idea that majority opinion should matter when deciding questions of life and death – must be rejected.

Flawed method Shah says the blowback thesis “is simple: Drone strikes kill more innocent civilians than terrorists which in turn radicalize affected populations and motivate them to join terrorist groups to retaliate against the United States.” According to him, the logic of the blowback thesis is that more civilians die than "militants" (let’s leave aside the ambiguity of this term for the moment), and the high ratio of civilian to militant deaths turn the affected population towards militancy.

Except, none of the examples he presents actually make this argument.

The 2013 Amnesty report to which he links for evidence has only one mention that approaches anything close to the kind of blowback thesis that Shah is putting forward. It quotes a Pakistani official who says that drones create a “thirst for revenge,” but that quotation makes no explicit link between the ratio of deaths and blowback. The rest of the 74-page document records specific stories of families affected by drones. In fact, it notes how militants have accused people of being American spies and killed them in retaliation for drone attacks, which is what you might call one version of blowback, though not the one Shah is interested in.

This was also the point made in the New York Times article, the second link in Shah’s piece. Here, Gen. James E. Cartwright, the former vice chairman of the Joint Chiefs of Staff and an adviser during Obama’s first term, stated that they were seeing blowback: “If you’re trying to kill your way to a solution, no matter how precise you are, you’re going to upset people even if they’re not targeted” (italics ours). In short, Cartwright says it doesn’t matter whether you’ve bombed the right person or the wrong person; you’re going to get blowback.

Finally, Shah quotes from an article by David Kilcullen and Andrew Exum. They come perhaps the closest to making the claim that Shah alleges is the core of the blowback thesis, but even here, it’s not entirely accurate. Here’s an expanded version of that quotation, with the section that Shah used in bold:

"Press reports suggest that over the last three years drone strikes have killed about 14 terrorist leaders. But, according to Pakistani sources, they have also killed some 700 civilians. This is 50 civilians for every militant killed, a hit rate of 2 percent — hardly “precision.” American officials vehemently dispute these figures, and it is likely that more militants and fewer civilians have been killed than is reported by the press in Pakistan. Nevertheless, every one of these dead noncombatants represents an alienated family, a new desire for revenge, and more recruits for a militant movement that has grown exponentially even as drone strikes have increased."

Kilcullen and Exum don’t establish a link between a higher ratio of civilian deaths to blowback. They simply write that regardless of what the actual ratios are — this is why the contrapositive “nevertheless” that Shah leaves out is so critical — family members of dead noncombatants may be moved to join the armed opposition. In fact, they aren’t discussing ratios, but rates. They argue that the rate of recruits into the militancy has increased even as drone attacks have increased. In other words, people are becoming militants at a faster rate than the ability of US forces to kill them. That argument, which is one actual version of the blowback thesis, doesn’t require a majority of any sort.

Shah follows a qualitative method and says that his sample was non-representative, meaning one can’t infer to the population. Yet, he couches his work in the context of statistically representative studies implying that his work and these studies are of the same species. They’re not. He presents the findings of those surveys, that is, the percentages of people who supported or opposed drones and follows that up by furnishing his readers with percentages from his own interviews, thus suggesting that they are comparable to a statistically representative study—they aren’t.

The issue isn’t bringing together different kinds of studies; the issue is allowing the reader to assume that they are all of the same kind when academics know that one reads qualitative work differently than quantitative studies.

He claims that because 79 percent of his respondents endorse drones, the blowback thesis isn’t supported. This is a strange conclusion, and it’s one that infers about the population based on a non-representative sample of interviews. Even if we pretend that this Frankenstein passes for a coherent argument, it still leaves 21 percent who oppose drone attacks and could potentially serve as recruits. It also fails to take into account that drone advocates may retract their support if they find their own family member killed.

Finally, the militancy has always been a tiny minority. It doesn’t take thousands to replenish or increase the numbers. Hundreds will do. That is why majority opinion has nothing to do with ascertaining whether there is blowback. The latter is better answered by examining the effects of drone bombardment and analyzing whether there has been blowback over the last 12 years.

Shah asserts that the voice of the local population is neglected, but dismisses a survey with a statistically representative sample by “a local NGO in FATA” that found that 63 percent of its respondents thought that “strikes are never justified.” He states that, when disaggregated, the survey shows that “support for drone strikes is the highest in North Waziristan, the FATA agency (district) where the CIA has carried out most of its lethal drone operations, compared to the other six.” This is misleading. He walks the reader into assuming that it is a majority or a near majority, but at 16.3 percent -16.1 percent saying it is “sometimes justified, if properly targeted and excessive civilian casualties are avoided” and only only 0.2 percent saying drones are “always justified”- it isn’t even close.

He doesn’t mention that 58 percent of the respondents from North Waziristan think that drone attacks are “never justified”; that goes up to 71.1 percent in neighboring South Waziristan, which has the second highest number of drone bombings. Shah curiously doesn’t name or link to the survey even though it’s available online and conducted by one of the larger FATA-linked organizations that has done such work.

Despite its lack of rigor, Shah’s work has been circulating in the press, on social media and in public discussions further sullying an already difficult debate. This is why the legitimacy lent to it by academics over at the Monkey Cage as well as the Washington Post is, at best, irresponsible. Both academics and journalists have failed us. The article isn’t academically sound, and given the misrepresentations of his opponents, it’s clear that the article was not even fact-checked. Such basic mistakes, at minimum, merit correction by editors.

Compounding the issue is that the media has a general love of numbers because they appear to provide a sheen of facticity to otherwise unsound claims. Indeed, drone advocates and the press are now repeating Shah’s unrepresentative numbers as if they were statistical evidence. The Pakistani newspaper Dawn went so far as to republish it as factual news in its “national” section. This is the imperial circuit of knowledge production that sanitizes and circulates justifications of American power as “fact.” Below, we briefly touch upon some of the assumptions behind these “facts.”

The limits of majority opinion

Countering Shah’s work by pointing to other surveys is not a road we wish to take. Though we have worked within the assumptions that underpin his work to show how it fails on its own logic, we don’t agree with them. For one, questions of life and death shouldn’t be adjudicated on the basis of majority opinion. Given that it’s unlikely that the US would have changed its decision to bomb people in Waziristan (and now, Balochistan) based on the opinions of Pakistanis, one can conclude that the opinion poll is a horrifying farce aimed at claiming “native” approval of the bombing.

Secondly, blowback is often a self-interested argument that assumes that the only problem with dropping bombs is whether you might have to suffer some consequences for it. That is what the US military and others mean when they talk about “blowback.”

What they don’t want to talk about is the political question of American imperial reach underpinned by a racialized demonization of Muslims that has it bombing mostly Muslim-majority countries. (It can simultaneously be true that there are militant organizations and that the US is bombing out of racist paranoia.)

The tragedy is that 147 interviews can potentially be a valuable resource for qualitative work. We’ve compiled interviews on FATA and Balochistan about the conflicts in each region; for the former, we now have a greater number than Shah, but we don’t draw hard and fast quantitative inferences from them. Instead, these interviews often undo the very categories through which much of the mainstream political discourse takes place.

People living in FATA, and particularly Waziristan, have suffered from the presence of brutal militant groups, the Pakistani military and American drones, together. Instead of thinking through the current political mire, the best some can apparently do is poll the people who endure by offering them macabre "choices" between being killed by the army or by drones.

Madiha Tahir and Mahvish Ahmad are co-founders of Tanqeed (www.tanqeed.org) and Ph.D. candidates at Columbia University and University of Cambridge, respectively.