I applaud and agree with Sarah Knuckey’s, Ole Solvang’s, Jonathan Horowitz’s and Radhya Almutawakel’s recent post (“Pentagon Admits Major Investigation Flaw: imeshey Rarely Talk to Air Strike Witnesses or Victims”) to the extent it suggests that more witness interviews would be helpful when investigating incidents involving civilian deaths, but their over-reliance on witness statements illustrates a lack of awareness about the limitations and hazards of what they call their “bedrock” fact-finding technique.

The writers of the post excoriate the Pentagon for not obtaining more eyewitness accounts during its civilian casualty investigations. Brig. Gen. Paul Bontrager, the deputy director of operations at U.S. Central Command, said during a recent briefing about an investigation into a civilian-casualty incident in the Syrian city of al-Jinah, that while the United States relies on a variety of investigation methodologies in investigating cases, it is “a rare thing with strikes like this that we can get on the ground in person, or that we can talk to anybody on the ground is not uncommon at all.”

The writers extrapolate from Bontrager’s comment about the particular investigative obstacles in a “strike like this” to make a broader condemnation. According to the writers, the “U.S. government’s failure to regularly interview witnesses is a critical flaw in their investigation methodology.” They insist that “[i]nterviewing is so essential that it has been described as the ‘bedrock’ of human rights fact-finding.”

Let’s be clear: Bontrager did not say that investigations never conduct on-the-ground interviews in civilian-casualty cases (consider the extensive investigation into the Kunduz hospital bombing, and the recent probe into the Mosul civilian-casualty incident), but rather that the investigators did not have access in “this” case to what he described (and the writers did not dispute) was a “confirmed and well-known al-Qaeda operational area.”

The Pentagon says that although it “did not have access to the scene” the “investigative team interviewed dozens of people.” In addition it “reviewed all available video and images, operational reports, and intelligence reports associated with the strike, while researching all regulations, standing operating procedures, commander’s guidance and other pertinent information.”

The writers fail to mention that at the briefing, Bontrager made clear that the investigation also “seriously reviewed information from a Human Rights Watch [HRW] report [“Attack on the Omar Ibn al-Khatab Mosque: US Authorities’ Failure to Take Adequate Precautions”] that came out recently concerning this strike.”

The investigating team used HRW’s report “to further assess if we could learn from their conclusions and their research.” Here’s the key: that report included information from 14 witness accounts, which HRW says they gathered. In addition, Bontrager said that:

We did reach out to the organizations that had published different documents, for example, the Human Rights Watch asking for any information that they had with regard to the strike. To again, try and get — gather any evidence that was out there and that offer still stands for Human Rights Watch, Syrian Observatory, anybody else who has something we would — we would be — we would welcome.

In other words, the investigation employed multiple information sources, and Knuckey, Solvang and Horowitz undersold the Pentagon’s incorporation of witness accounts in its assessment.

Still, in light of modern investigative techniques, human rights groups may want to rethink the extent to which witness interviews ought to form the “bedrock” of their investigations. The article the writers cite, which describes witness statements as the “bedrock” of investigations, was published in 1990. Much has changed with respect to investigative techniques in the intervening decades, and research also shows that witness accounts are often unreliable.

For example, in 2005 Jennifer Overbeck observed in her New York University Law Review article that: “[T]hirty years of psychological research into the workings of human memory have revealed that eyewitness accounts are frequently flawed, either because the witness’s original perception of the event was flawed, or because the memory was subconsciously altered…”

Likewise, a 2010 Scientific American article — “Why Science Tells Us Not to Rely on Eyewitness Accounts” – concludes that scientific study demonstrates that “eyewitness testimony is fickle and, all too often, shockingly inaccurate.” Today, the Innocence Project says that “[e]yewitness misidentification is the greatest contributing factor to wrongful convictions proven by DNA testing, playing a role in more than 70 percent of convictions overturned through DNA testing nationwide.”

In a 2012 TED talk – “Why eyewitnesses get it wrong” – forensic psychologist Scott Fraser explains why even the best intended witnesses can be mistaken.

Under the best of observation conditions, the absolute best, we only detect, encode and store in our brains bits and pieces of the entire experience in front of us, and they’re stored in different parts of the brain. So now, when it’s important for us to be able to recall what it was that we experienced, we have an incomplete, we have a partial store, and what happens? Below awareness, with no requirement for any kind of motivated processing, the brain fills in information that was not there, not originally stored, from inference, from speculation, from sources of information that came to you, as the observer, after the observation. But it happens without awareness such that you don’t, aren’t even cognizant of it occurring. It’s called reconstructed memories.

Professor Knuckey and her co-writers concede that “eyewitnesses may have poor recall, they may be biased, they can intentionally falsify” but they say these challenges can be overcome by training in interviewing techniques and by interviewing “large numbers” of witnesses, as well as efforts to “corroborate witness statements with other sources of information, such as site visits, physical evidence, photographic evidence, medical reports, hospital records, and so on.”

This may be true to some extent, but the results are not always encouraging. Consider Amnesty International’s 2013 report “”Am I next?” U.S. Drone Strikes in Pakistan.” It featured what one would presume to be a young Pakistani girl on the cover, but buried in small print in the endnotes, Amnesty International said, it

“tried to interview women and girls, as well as men and boys, in order to obtain a balanced picture of events and to assess whether gender plays a role in these issues. Our access to women, and girls in particular, was, however, quite limited. Women and girls in the Tribal Areas region face particularly severe restrictions on both their movements and their communications with individuals outside the family unit.”

By definition then, the virtual exclusion of an entire gender – and particularly the one suffering the most under extremism – makes their report, by its own terms, not a “balanced picture.” What is more is that this omission indicates how witness interviews are controlled by local powers (a problem not unique to Pakistan).

There are other reasons for concern. In analyzing an Amnesty International investigation, David Axe points out in his essay (“Dear Amnesty International, Do You Even Know How Drones Work?”) that “the group’s claims are based on only the most cursory evidence and potentially unreliable eyewitness accounts.” Why? Axe argues that some statements are “inconsistent with known drone tactics and the well-understood limitations of unmanned aircraft in general.” He adds:

The attackers could have been manned warplanes, and Pakistani rather than American…It seems that some Pakistanis in the tribal areas describe all warplanes and even helicopters as “drones”—and as American. They apparently just assume that explosives raining from the sky necessarily come from robotic planes flown by the U.S.

Similarly, Joshua Foust critiqued a HRW investigative report, “Between a Drone and al-Qaeda,” about the alleged death of 57 civilians in six strikes. In his essay, “How Human Rights Groups Misinterpret Drone Strikes,” Foust observed that notwithstanding HRW’s report’s title, the incident that caused most of the reported deaths (41 of the 57) “wasn’t even a drone strike” (it was a cruise missile operation). Moreover, he concludes that “[a]nother strike [HRW] profile[s] even seems to have been carried out by Yemeni jets, not American drones.”

More recently, in HRW’s al-Jinah report, the group says it “interviewed by phone 14 people” who they claim had “first-hand knowledge of the attack.” If HRW is depending upon witnesses they reach by phone in a war zone that is a “confirmed and well-known” area of terrorist operations, is there any reason to be skeptical? Of course there is.

This particular strike took place in an area controlled by al-Qaeda, where we don’t know who has access to phones. In ISIS-controlled areas in Syria, “militants punish people who use the internet and mobile phones, fearing that they may provide intelligence to their enemy,” the Guardian reported in 2015. In March of this year, an NGO described that for those under ISIS rule, there are “No cell phones. No internet. Being caught with a cell phone was a crime punishable by death under ISIS—an effective means of control in a landscape like Iraq.”

How can we be sure that those who human rights organizations call up are not simply under effective terrorist control? After all, few dispute that terrorists these days have an explicit strategy of staging incidents to falsely blame on coalition forces.

Given that the mere possession of a phone – let alone contact with outsiders – can be punishable by death, serious ethical concerns are raised by even attempting to contact civilians in these areas. Moreover, do human rights groups really want the U.S. to launch a full-blown military operation into a terrorist-infested area – with potential human costs on all sides, which such an investigation could require, simply to be able to say that they (attempted?) to interview witnesses on-the-ground?

There are other potential biases at play too. Dr. Christopher Swift, a lawyer/scholar with much on-the-ground experience in Yemen, noted in an email to me that a 2015 Open Society report (“Death by Drone: Civilian Harm Caused By U.S. Targeted Killings in Yemen”) was one where “an international human rights organization that opposes drone strikes hired a Yemeni human rights organization that also opposes drones strikes to do field work on targeted killings.” While Dr. Swift is “all for reporting on screw-ups,” the reality is that “ultimately, there are few ground-level sources in the places where AQAP is most active.”

Another concerning feature of these types of reports is the issue of transparency. Professor Knuckey and her co-writers write that “[s]peaking to victims, witnesses, and other people on the ground also helps provide these investigations a degree of transparency.” Maybe so. But consider that the Open Society report admits:

The interviewees were informed of the purpose of the interview and asked if they consented to their identities being disclosed in this report. The report does not provide identifying information for interviewees who were not comfortable with being identified.

In other words, there is no way to independently verify the statements, or even determine if the witnesses actually exist. The U.S. Supreme Court has noted that many factors can impact the reliability of a witness’ statement, and the inability to confront a witness is a central reason the Court rejects testimonial hearsay, even when the sources are clearly identified. But testimonial hearsay – from unidentified sources –is sometimes relied upon in human rights organizations’ investigations.

It is commendable that human rights investigators are “increasingly using satellite imagery techniques and other technologies,” as Knuckey, Solvang and Horowitz write. It should be noted that the military has been using satellites for decades. I believe human rights organizations are deluding themselves if they think they have equipment and resources on par with the U.S. military’s sophisticated process for battle damage assessment.

As the al-Jinah briefing indicates, some of these capabilities are classified, but can often involve extended coverage by drones. The writers are dismissive of the value of full motion video recordings of the actual incident, but that may be because drones are a technology they do not possess, or if they do, they do not use them on a scale anywhere approaching that of the U.S. military.

In any event, as the Washington Post reported recently, the surveillance before and even after the strike may go on for weeks, and involve “dozens of U.S. military and intelligence officials around the world.” The New York Times similarly reported last May about the high-tech command center “crammed with liaison officers from countries in the American-led coalition, the American military services, intelligence experts and officers who plan and direct” what the Times calls “the complex ballet of strike, surveillance and refueling aircraft that keeps the war going around the clock.” All of this gives the military remarkable (albeit certainly not perfect) situational awareness that is only achievable with costly technology expertly employed.

The U.S. also possesses audio surveillance capabilities. While human rights groups call what witnesses (who agree to cooperate with them) choose to share the “bedrock” of their investigations, is it not plausible that a greater degree of accuracy might be obtained from what those on the ground choose to tell each other, as opposed to an outside interviewer? No human rights organization possess these kinds of signals intelligence resources.

To be clear, interviewing witnesses at the site of an alleged civilian-casualty incident is helpful and should be done whenever possible, so long as doing so doesn’t put the interviewers or interviewees at unnecessary risk. Investigations by human rights groups – to include their witness interviews – should be, as was the case here, “seriously reviewed.” Indeed, Pentagon investigations should welcome data from all bona fide sources (to include the media as was, again, the case here).

But the writers’ criticisms go too far when they suggest that the U.S. is failing to meet its obligations to “account for its possible violations of international legal obligations.” In my view, the human rights community needs to be more conscious of the limits of eyewitness testimony, but also of the value of modern investigative technologies and approaches that they too quickly dismiss without sufficient reason.

Image: Defense Department officials released this aerial photograph taken after a March 16, 2017, U.S. strike in Jinah, Syria. U.S. Defense Department photo.