Illustration by Tom Bachtell

Last week, Americans got a reminder that it sometimes takes lawbreakers with a conscience to bring to light more profound violations. On March 8, 1971, burglars broke into an F.B.I. field office in Media, Pennsylvania, and stole hundreds of the agency’s files. Nearly forty-three years later, several of them owned up to what they’d done, first in “The Burglary,” a new book by Betty Medsger, a former Washington Post reporter, and then in interviews with the Times. The instigator of the break-in, it turns out, was a Haverford College physics professor named William Davidon, who spoke extensively with Medsger before he died, last November. Among the others who came forward were a married couple, John and Bonnie Raines. In 1971, he was a professor of religion at Temple University, she was a grad student studying child development there, and they were raising three young children. Though the couple seemed unlikely suspects, they were part of the Quaker- and Catholic-influenced peace movement that flourished in and around Philadelphia at the time, and which sustained the activities of the Berrigan brothers, among others. The burglars believed that the antiwar movement was being infiltrated by F.B.I. informants, and that getting hold of agency documents was the only way to prove it—though this was the first and last time they would do anything so extreme in that cause. None of them suspected that J. Edgar Hoover’s dirty-tricks campaign was designed to neutralize almost all forms of political dissent.

Choosing the night of the Muhammad Ali–Joe Frazier heavyweight-championship bout for the break-in, the burglars counted on most people being distracted—their TVs turned up loud—including those who lived in the building that housed the F.B.I. office. They got away with suitcases full of files. At a farmhouse where Davidon and his gang of pacifists met to examine their haul, one of the first documents they read instructed agents to “enhance the paranoia” by dogging student and antiwar activists, and to “get the point across there is an F.B.I. agent behind every mailbox.” Other files, which they began releasing to reporters (holding back those which they thought would endanger crime-fighting operations), showed that Hoover’s F.B.I. had a twisted idée fixe about African-Americans; it aimed to put spies in every black student union at every college in the country, “without regard for whether there had been disturbances on such campuses,” and had largely achieved that goal. As Medsger writes, the over-all impression the files gave was that Hoover and many other F.B.I. officials “thought of black Americans as falling into two categories—black people who should be spied on by the F.B.I. and black people who should spy on other black people for the F.B.I.”

But that, as it happened, was not the worst of the revelations. One of the files contained a routing slip with a strange word on it: COINTELPRO. It took two years and the determination of Carl Stern, an NBC News reporter who filed several Freedom of Information Act requests and a lawsuit against the Justice Department and the F.B.I., for Americans to discover what COINTELPRO was. Since 1956, Hoover had been running a highly secret program under that name which not only spied on civil-rights leaders, suspected Communists, public critics of the F.B.I., student activists, and many others but also sought to intimidate, smear, and blackmail them, to break up marriages, get people fired, demoralize them. These were the auspices under which the F.B.I. spread a false rumor that the actress Jean Seberg, who had recently given a donation to the Black Panther Party, was pregnant by a Panther leader. (Seberg was pregnant, and, shortly after the story appeared in a gossip column, she miscarried.) It was under COINTELPRO that Hoover waged his assault on Martin Luther King, Jr., wiretapping his hotel rooms and recording his sexual encounters, and, at one point, trying to coerce him to commit suicide.

None of this, it seems clear, would have come out—or, at least, not in time for anything to be done about it—had it not been for the purloined files. Because it did come out, Americans could have a public debate about Hoover’s agency, and its equating of political activism with criminal activity. The files and the investigative journalism that followed up on them led to the Church Committee hearings, and to reforms that helped rein in the agency’s worst excesses: a ten-year term limit for the F.B.I. director (Hoover had run the agency since 1924); permanent congressional oversight of the F.B.I.; and the creation of the fiSA (Foreign Intelligence Surveillance Act) courts, which since 1978 have reviewed warrants for conducting certain electronic surveillance. As Frederick A. O. Schwarz, Jr., the chief counsel to the Church Committee, said, the COINTELPRO activities “undermined American democracy, violated the law and subverted the Constitution.” A smaller crime had allowed us to check a much greater one.

In most respects, the National Security Agency’s collection of domestic phone records which Edward Snowden revealed is nowhere near as disturbing as COINTELPRO’s activities. It is neither ideologically motivated (the N.S.A.’s actions were initially ramped up in response to a real attack; Hoover’s were intent on destroying perceived enemies) nor thuggish (it entails surveillance but not infiltration or harassment or blackmail or smear campaigns). Yet in one regard—its technological prowess—it is worse. As the U.S. District Court Judge Richard Leon wrote last month, in an opinion that strongly suggests that the metadata collection could be found unconstitutional, “Records that once would have revealed a few scattered tiles of information about a person now reveal an entire mosaic—a vibrant and constantly updating picture of a person’s life.” Leon noted that the government did not cite any instances in which the data collection proved necessary in preventing an imminent attack, and concluded that, when weighed against the “almost-Orwellian technology that enables the Government to store and analyze the phone metadata of every telephone user in the United States,” the N.S.A.’s rationale was simply too weak.

Snowden’s leaking of the N.S.A.’s data-mining has something fundamental in common with the COINTELPRO story, too. Without that drastic action, we would not have the findings of a Presidential review board composed of legal and intelligence experts, which last month cautioned that the government’s storage of metadata poses a threat to civil liberties, personal privacy, and public trust, and offered President Obama a series of recommendations for fixing the breach, adding that “free nations must protect themselves, and nations that protect themselves must remain free.” Three months before Snowden acted, James Clapper, the director of National Intelligence, was asked at a congressional hearing whether the N.S.A. collected data on American citizens. He replied that it did not. In retrospect, it’s easy to say that we would have found out about the practice by other, more respectable means. Unfortunately, it doesn’t often work out that way. ♦