Errol Morris on photography.

Read Part 1; Part 2; Part 3.

4.

ABSENCE OF EVIDENCE

ISN’T EVIDENCE OF ABSENCE

Photo

The phrase “absence of evidence isn’t evidence of absence” has been attributed to Martin Rees, O.M., astronomer royal, former master of Trinity College, and ex-president of the Royal Society. [1] It was used by him and then by Carl Sagan in discussions about the possibility of intelligent life somewhere else in the cosmos. (There are those who believe we have no direct evidence of extraterrestrial intelligence and those who believe we might have direct evidence, e.g., “The Wow! Signal,” a strong signal detected by a radio telescope that has never been repeated.)[2] And it involves questions about the nature of evidence, about how to separate in a mass of data the signal from the noise.



I have met Martin Rees a number of times, most recently at Stephen Hawking’s 70th-birthday party, and so I called him hoping to get some additional insight into his use of the phrase.

ERROL MORRIS: The absence of evidence is not evidence of absence — every time I’ve tried to track down its origins, it leads back to you.

MARTIN REES: I’ve used it, but I’m sure I wasn’t the first to use it.

ERROL MORRIS: Does it appear in any of your books or writings?

MARTIN REES: It certainly appears in some of my written lectures. [The citations are few and far between. But it was attributed to Martin Rees in a symposium on extraterrestrial intelligence, Life Beyond Earth and the Mind of Man, edited by Richard Berendzen.[3]]

ERROL MORRIS: Even if you didn’t originate it, I was hoping you would explain to me what you meant by it.

MARTIN REES: I don’t think it’s anything profound. It’s within the context of looking for aliens — if we don’t see anything, it doesn’t mean they’re not there. They may be very different from us; they may not be trying to communicate. When I’ve used it, it’s just been in that specific sense.

ERROL MORRIS: Well, I think what happened is — and I could be completely wrong about this — that it appeared initially in the context of, “Is there life elsewhere in the universe?” And it’s not just an absence of evidence, because we have all kinds of statistical evidence that would suggest that there is.

MARTIN REES: No, we don’t. We don’t have any evidence.

ERROL MORRIS: Yes and no. We have statistical evidence. We have evidence of how many planets that might be like ours? For example, the Drake equation [the equation that estimates the likelihood of life elsewhere in the universe] [4] —

MARTIN REES: Yes. But we don’t know if life started in these places. Certainly, at the time of the Cyclops [Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life, NASA, 1971], they were looking for signals from intelligent life. And even if simple life is common, then intelligent life may be rare. And only a tiny subset of intelligent life might be sending out signals.

ERROL MORRIS: But Rumsfeld, when he used it —

MARTIN REES: I didn’t know he had used it.

ERROL MORRIS: He used it again and again. He used it in the case of weapons of mass destruction in Iraq. Presumably, the argument is, “Just because we can’t find any evidence that Saddam Hussein has weapons of mass destruction, that doesn’t mean he doesn’t have weapons of mass destruction.”

MARTIN REES: I don’t regard that as profound. It’s a fairly obvious comment that is used in all kinds of different contexts.

ERROL MORRIS: And the known known and the known unknown?

MARTIN REES: This is not profound, either, but it’s an interesting distinction. Most people who write about risk and hazard would distinguish between risk and uncertainty.

ERROL MORRIS: Yes, but —

MARTIN REES: Risk is something where you can assign a probability. But as to what’s going to happen in 10 or 20 years, then you can’t really assign realistic probabilities. I suppose that’d be what you’d call an “unknown unknown.” Ultimately, I don’t think there’s anything special in the scientific method that goes beyond what a detective does.

I believe Martin Rees is correct. There is no direct evidence of intelligent life elsewhere in the universe, though we have statistical evidence to suggest that it might be out there. We have theories about where to look for it, how we might detect it, how rare it might be, and so on. We have charts and equations and tools and algorithms to help direct our focus.[5] But is combing the cosmos for signs of intelligent life really just like inspecting Iraq for signs of W.M.D.?

Photo

Martin Rees wrote in Our Cosmic Habitat:

Some brains may package reality in a fashion that we can’t conceive. Others could be uncommunicative: living contemplative lives, perhaps deep under some planetary ocean, doing nothing to reveal their presence. There may be a lot more life out there than we could ever detect. Absence of evidence is not evidence of absence.[6]

Absence of direct evidence, may not be proof of absence, but often absence of evidence is evidence of absence. If a weapons inspector looks for W.M.D. in a building and finds nothing, is that absence of evidence? Or evidence of absence? I would argue that it is evidence of absence. Rumsfeld took a principle used in one context — is there intelligent life elsewhere in the universe — and applied it to Iraq. Imagine someone tells you that there is an elephant in the room. You search the room, opening drawers, checking closets, looking under the bed. No elephant. Absence of evidence or evidence of absence?

Hans Blix, the executive chairman of the United Nations Monitoring, Verification, and Inspection Commission, addressed some of these concerns in his book Disarming Iraq:

Several countries, including the U.S., had given us a good number of sites for possible inspection, and at none of the many sites we actually inspected had we found any prohibited activity. The sites we had been given were supposedly the best that the various intelligence agencies could give. This shocked me. If this was the best, what was the rest? Well, I could not exclude the possibility that there was solid non-site related intelligence that was not shared with us, and which conclusively showed that Iraq still had weapons of mass destruction. But could there be 100-percent certainty about the existence of weapons of mass destruction but zero-percent knowledge about their location?[7]

Blix leaves us with a rhetorical question. Could there be 100 percent certainty of the existence of W.M.D. in Iraq and no knowledge of where they might be? Blix’s answer is — probably not. In a news conference on Sept. 26, 2002, Pam Hess asked Rumsfeld:

PAM HESS: If an inspection team goes in now and finds nothing, because perhaps Iraq is very good at hiding it, or perhaps they have nothing — but you all are of the belief that they have it. If they find nothing, does it make your job more difficult in trying to assemble an international coalition to disarm him by other means?

DONALD RUMSFELD: Goodness gracious, that is kind of like looking down the road for every conceivable pothole you can find and then driving into it. I don’t get up in the morning and ask myself that. We know they have weapons of mass destruction. We know they have active programs. There isn’t any debate about it [my emphasis].

Ultimately, W.M.D. were not found in Iraq, and Rumsfeld’s belief proved to be false. Many people assume that Rumsfeld was lying. That he knew that there were no W.M.D. in Iraq. But I believe that he was able to convince himself that he was telling the truth. Or even worse, that he lacked the ability to discriminate between truth and fantasy. That over the years he had developed a gobbledygook philosophy that — seemingly paying lip service to empiricism — devalued evidence and made a mockery of logic. Alas, if you believe that you are 100 percent right, then your beliefs are like a hard, impenetrable, protective shell, like the carapace of a turtle. [8]

What do I take from this? To me, progress hinges on our ability to discriminate knowledge from belief, fact from fantasy, on the basis of evidence. It’s not the known unknown from the known known, or the unknown unknown from the known unknown, that is crucial to progress. It’s what evidence do you have for X, Y or Z? What is the justification for your beliefs? When confronted with such a question, Rumsfeld was never, ever able to come up with an answer.

Photo

The history of the Iraq war is replete with false assumptions, misinterpreted evidence, errors in judgment. Mistakes can be made. We all make them. But Rumsfeld created a climate where mistakes could be made with little or no way to correct them. Basic questions about evidence for W.M.D. were replaced with equivocations and obfuscations. A hall of mirrors. An infinite regress to nowhere. What do I know I know? What do I know I know I know? What do I know I don’t know I don’t know? Ad infinitum. Absence of evidence could be evidence of absence or evidence of presence. Take your pick. An obscurantist’s dream.

There’s a quotation I have never liked. It comes from F. Scott Fitzgerald’s The Crack-Up. “The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function.” Not really. The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time and know they are opposed.

People embrace contradictory positions, often without being aware of it. Sometimes not caring. Sometimes proud of it. Rumsfeld seems (with pleasure) to say “p” and “not-p.” What he would call the two sides of the coin. One side: “If you wish for peace, prepare for war.” The other side, “Belief in the inevitability of conflict can become one of its main causes.” Not exactly a contradiction. But where does he stand? His follow-up: “All generalizations are false, including this one” — doesn’t clarify much of anything.[9]

When asked how Colin Powell could have presented such shoddy evidence for W.M.D. in Iraq to the United Nations, Rumsfeld told me, “…because he believed it.” Fine, as far as it goes. My guess is Rumsfeld is right. When Powell appeared before the United Nations on Feb. 5, 2003, he believed what he was saying.[10]

The secretary also showed satellite photographs of what he said were chemical and biological facilities, and drawings based on witnesses’ descriptions of trucks and rail cars converted into mobile laboratories for lethal materials, allegedly intended to evade detection. He said various records and intelligence showed that Mr. Hussein was making nuclear weapons and developing rockets and aircraft to deliver all his weapons.[11]

Rumsfeld, too, may believe what he is saying. But believing something does not make it true. The question is why he believed what he believed. On the basis of what evidence? Mere belief is not enough.

In Lewis Carroll’s Alice in Wonderland, Alice, perplexed by her encounter with the Cheshire Cat, says, “I have seen a cat without a grin, but I have never seen a grin without a cat.” I had a similar experience with Donald Rumsfeld — his grin and my puzzlement about what it might mean. I was left with the frightening suspicion that the grin might not be hiding anything. It was a grin of supreme self-satisfaction and behind the grin might be nothing at all.



Photo

[1] In a somewhat different form it appears in the writings of William Cowper, although I have been hard-pressed to find any specific attribution. But Cowper’s version involves proof, not evidence. That absence of proof isn’t proof of absence. Just because we haven’t been able to prove that God exists doesn’t mean that he doesn’t. Samuel Joeckel writes in an article for Quodlibet journal entitled “Localizing the Problem of Evil: William Cowper and the Poetics of Perspectivalism”:

Throughout his life, Cowper suffered from psychological disorders and traumas that racked his tempestuous mind and soul. In his provocatively titled study, Boswell’s Clap and Other Essays: Medical Analyses of Literary Men’s Afflictions, medical doctor William B. Ober diagnoses Cowper as a “psychotic who suffered from mental depression with suicidal tendencies. His madness was colored strongly by religious delusions centering about his own damnation.”

[2] On the night of Aug. 15, 1977, an extraordinary signal was detected by the Big Ear radio telescope at The Ohio State University. Dr. Jerry Ehman — reviewing the recent computer printouts recording what Big Ear had “heard” — was astonished to see that the telescope had detected a strong, narrowband radio signal that seemed to fit the pattern researchers expected to see if intelligent life attempted to make contact.

The alphanumeric sequence circled in the Wow! Signal indicates the intensity variation of the signal as received at the Big Ear radio telescope, and functions as a sort of signal-to-noise ratio. Each number or letter indicates the number of standard deviations (A=10, B=11, etc.) by which the received signal exceeded the average background noise. These numbers and letters tell us both that the signal received was way louder than the noise of space, and also that it increased and faded in intensity in a manner consistent with a transmission received from an extraterrestrial source. That is to say, it came from outer space, it was loud, and it lasted for as long as we could measure. Alas, it was never detected again.

[3] Berendzen writes:

A generation ago almost all scientists would have argued, often ex cathedra, that there probably is no life in the universe besides what we know here on Earth. But as Martin Rees, the cosmologist, has succinctly put it, “absence of evidence is not evidence of absence.”

[4] The Drake equation calculates N, the number of civilizations in our galaxy with which radio-communication might be possible based on—

R* 1 = the average rate of star formation in our galaxy

f p = the fraction of those stars that have planets

n e = the average number of planets that can potentially support life per star that has planets

f l = the fraction of planets that could support life that actually develop life at some point

f i = the fraction of planets with life that actually go on to develop intelligent life (civilizations)

f c = the fraction of civilizations that develop a technology that releases detectable signs of their existence into space

L = the length of time for which such civilizations release detectable signals into space.

The last variable “L” estimates how long before civilizations blow themselves up or go extinct.

[5] The chart depicts the terrestrial microwave window — the range of frequencies at which SETI Researchers expect to be able to intercept a signal coming from another planet. Between 1 and 10 GHz, a signal can travel through both outer space and the Earth’s atmosphere without much interference. Above 10 GHz, water and oxygen in Earth’s atmosphere will begin to interfere with the reception of the signal. The range from 1 to 10 GHz comprises about 9 billion channels, each of which can be monitored from Earth, and one of which we might expect alien life to utilize if they want to broadcast across the cosmos.

[6] Martin Rees, Our Cosmic Habitat, Princeton University Press, 2001, p. 28.

[7] Hans Blix, Disarming Iraq, Pantheon, 2004, p. 156.

[8] Bayes’ Theorem doesn’t apply if you are 100 percent convinced you are right. Contrary or supplementary evidence—no matter how well supported—will have no effect on that conviction.

[9] “All generalizations are false, including this one” leads logically to “Some generalizations are true.” If you wish to trace this error back, consider “All Cretans lie,” uttered by a Cretan. It can’t be true, but it can be false. What’s interesting is that it not only leads to “Some Cretans tell the truth,” but it also leads to the conclusion that the Cretan speaking is not one of them.

[10] Colin Powell from Sept. 8, 2005:

I looked at the four [sources] that [the C.I.A.] gave me for [the mobile bio-labs], and they stood behind them, … Now it appears not to be the case that it was that solid. At the time I was preparing the presentation, it was presented to me as being solid, April 3, 2004.

[I felt] terrible … [giving the speech] … It’s a blot. I’m the one who presented it on behalf of the United States to the world, and [it] will always be a part of my record. It was painful. It’s painful now.

[11] The New York Times, “Powell, in U.N. Speech, Presents Case To Show Iraq Has Not Disarmed,” Feb. 6, 2003.

ACKNOWLEDGEMENTS

My thanks to the many people who worked on this essay. Ann Petrone and Charles Silver helped with editing and style. James Maxwell Larkin and Josh Kearney did research. Yvonne Rolzhausen fact-checked and worked on editing and style. Karen Skinner and Molly Rokosz assisted in proofing. I have benefited from discussions with Mark Danner, whose three-part series on Donald Rumsfeld appeared in The New York Review of Books. The entire series can be found at //www.markdanner.com. And to my wife, Julia Sheehan, who when asked to compare Robert McNamara and Donald Rumsfeld, called McNamara “The Flying Dutchman,” destined to travel the world looking for redemption and never finding it; and called Rumsfeld “The Cheshire Cat.” My production designer Ted Bafaloukos provided his version of John Tenniel’s illustration from Alice in Wonderland.