Did the United States have a hand in assassinating Congolese and Dominican leaders in 1961? What did President Richard Nixon’s White House know about a successful plot to kill the head of the Chilean army in 1970? After the Cold War ended, did top U.S. military commanders retain the authority to strike back if a surprise nuclear attack put the president out of commission?

The answers to these and other historical mysteries are likely knowable—but they are locked in presidential libraries and government archives and inaccessible to researchers. The reason: the U.S. government’s system for declassifying and processing historical records has reached a state of crisis. Congress has failed to adequately fund the parts of the government charged with processing records, resulting in understaffed offices and years-long backlogs. At the same time, some agencies responsible for declassifying documents have deliberately dragged their feet and erred on the side of needless secrecy.

Declassification is vital to a thriving democracy. Not only does it help the public hold leaders accountable; it also allows for a more accurate and comprehensive accounting of the past.

Without declassification, the American hand in the coups in Iran in 1953 and Guatemala in 1954 would have remained hidden, and Americans would have no way of understanding their country’s Third World meddling. The exposure, through declassification, of the CIA’s assassination plotting in the late 1950s and early 1960s produced public pressure that resulted in executive orders banning political assassinations. Declassified documents brought to light the intelligence failures during 1962 that delayed the detection of Soviet missile deployments in Cuba.

For every secret revealed, however, untold numbers are still hidden away. Only by unsealing its archives can the United States live up to its ideals as an open society and learn from its past.

THE LONG AND WINDING ROAD

By law, the classified documents that most federal agencies produce in the course of their work—from State Department cables to Pentagon policy papers to White House e-mails—must be preserved and eventually transferred to the National Archives or the presidential libraries that it runs. Some agencies, such as the CIA, hold on to their records indefinitely.

Whether they are marked confidential, secret, or top secret, classified records remain classified by default; there is no rule requiring declassification after a given number of years. The National Archives does review old records for full or partial declassification, often in response to requests from the public. But the power to declassify a document lies in the hands of the agencies involved in its creation or interested in its contents. All the agencies with a stake in a document must sign off on its declassification—and any one of them, even if it is not the agency where the document originated, can block it.

All the agencies with a stake in a document must sign off on its declassification—and any one of them can block it.

If a request for declassification is denied, the story does not end there. For documents requested under the Freedom of Information Act, requesters may file an administrative appeal and ultimately a lawsuit, although few choose to undertake such an expensive and time-consuming process. Since the 1970s, executive orders have provided another avenue, called a “mandatory declassification review” request, through which one can ask for classified records that can be specifically identified. For this type of request, the final resort is to file an appeal with a body composed of representatives from various agencies, called the Interagency Security Classification Appeals Panel.

The whole process takes an unconscionably long time. I am still awaiting final decisions from the National Archives for State Department records from 1969 concerning U.S. policy toward the Israeli nuclear program. I initially made the request in 2005. Nine years later, the National Archives responded by informing me that the Defense Department had denied 31 documents in their entirety. My appeal is still pending. Many historians can’t afford to wait: 14 years is far longer than the lifespan of most research projects.

PAPER JAM

Nearly every federal office through which a declassification request must pass is simply overwhelmed. Agencies handle requests for records that they retain, and few devote many resources to processing them. For records held at the National Archives, requests are coordinated by the National Declassification Center, a small office in a significantly underfunded agency that is reluctant to challenge the rights claimed by other agencies. The Interagency Security Classification Appeals Panel, for its part, has been relatively effective in reversing unreasonable decisions made by agencies, but it also has a woefully small staff: just three full-time employees.

Things don’t look much better at the presidential libraries. They used to systematically review classified presidential records, file by file, but this undertaking has collapsed. The CIA once supported a process for scanning classified files at the libraries so that they could be reviewed in Washington by the interested agencies, but it has stopped providing essential funding. Huge backlogs of classified files at presidential libraries have emerged. At the Kennedy Library, even the review of White House tapes, records capturing the kind of firsthand evidence that historians dream of, has ground to a halt. To help ease the logjam, the National Archives has begun to centralize classified presidential records in Washington, D.C., but it is questionable whether it has the resources to systematically review them.

Antiquated practices abound. Some agencies lack the technology to electronically transfer classified information to other agencies for review. They use couriers or the mail instead. Others lack databases indicating what they have declassified in the past. Often, agencies have unwittingly classified information that has already been released: in 2019, to cite just one example, the Defense Department excised much of a 1963 Joint Chiefs of Staff document concerning security guarantees for Israel that had been published in its entirety in 1995 by the State Department.

The National Archives is unequipped to wade through the terabytes of classified electronic records it has been accumulating.

The National Archives is unequipped to wade through the terabytes of classified electronic records it has been accumulating—nearly 200 million PDFs, emails, and other digital files spanning decades, from the Ford administration to the Obama years. These include countless tantalizing nuggets, such as the infamous messages sent over the Reagan White House’s e-mail system that senior officials involved in the Iran-Contra affair tried to delete but investigators recovered.

Over time, the mountain of digital files will only grow as agencies turn over their classified electronic materials. Further compounding the problem, in an effort to make digital records the norm, the National Archives has announced that beginning in 2022 it will no longer accept paper records. This policy is particularly worrisome for those interested in CIA’s enormous trove of paper files dating back to the 1940s.

The growing digital backlog—on top of the analog one—presents the National Archives with a formidable task. Yet the National Archives’ budget for 2020 is just $358 million—virtually the same, in real terms, as it was in the early 1990s. Starved for resources, the National Archives has stopped providing some of the services it once did. For example, it no longer creates finding aids for newly opened collections, identifying them only with bare-bones information. Releasing archival material without finding aids is akin to publishing a book without a table of contents, chapters, or an index.

TOO SECRET

If only a lack of resources were the end of the problem. But certain parts of the government also resist declassification as a matter of course. While some agencies tend to make reasonable decisions about documents, others do not. The State Department has a good track record, whereas intelligence agencies such as the CIA and the National Security Agency act predictably by denying many requests. Congress has given the intelligence agencies wide leeway in excluding their operational files, which can involve espionage, covert action, and communications intelligence, from declassification requests. Thus, important aspects of U.S. history will remain secret indefinitely, unless the White House or Congress intervenes.

In the past, the CIA’s Historical Review Panel, an independent group of academics, nudged the agency toward greater transparency. But this January, the panel’s chair announced that the CIA had put the group on hiatus without explanation. The suspension was of a piece with a disturbing trend: in 2017, the agency virtually halted its program of reviewing records more than 25 years old.

Yet the intelligence agencies can do the right thing, when prodded. In 2016, in response to requests from the Argentine government and human rights activists, the Obama White House ordered the intelligence community to begin a large-scale declassification project concerning Argentina’s so-called Dirty War, the campaign of torture and killings undertaken by the military junta that ruled the country from 1976 to 1983. The project, which continued under the Trump administration, resulted in the release of some 15,000 documents. Alas, such focused declassification projects create another problem: because there is no budget for them, agencies finance them by shifting scarce resources away from regular declassification work.

The Pentagon is the poster child for overclassification and delay. Since late in the George W. Bush administration and accelerating during the Obama years, reviewers at the Defense Department have routinely refused to declassify information that they allege would, in the words of an executive order, “cause serious harm to relations between the United States and a foreign government, or to ongoing diplomatic activities of the United States.” Never mind that the documents denied often come from the State Department—or, for that matter, that diplomatic relations are the province of that agency, not the Pentagon.

Unlike the CIA and the State Department, the Defense Department has a chaotic, decentralized classification review process. Reviewers are required to consult higher-level officials in various policy offices who lack the frame of mind, experience, and expertise needed to review historical documents objectively. Officers working on declassification at the Pentagon’s Joint Staff contribute to the chaos. Career-minded and fearful of making mistakes, they tend to take the path of least resistance—denial—when evaluating requests.

The Defense Department’s pernicious influence extends to the Foreign Relations of the United States series, the indispensable documentary history published by the State Department. In 2018, the State Department’s Historical Advisory Committee, the body that oversees its publication, reported that the Defense Department “negligently” and “egregiously violated” the State Department’s Congressionally mandated work of producing “thorough, accurate and reliable” volumes by denying in their entirety nearly 600 documents that the editors had selected. And yet the Pentagon has done nothing to address these serious concerns.

OPEN UP

To count on the Trump administration, currently consumed by other matters, to reform declassification procedures would be foolish. Nor have legislators shown much interest in declassification: under both Democratic and Republican control, Congress has kept the National Archives on low rations, and there are few votes to be gained from pushing for less secrecy.

If it wanted to, however, Congress could alleviate the crisis in declassification. It could start by increasing the budget of the National Archives, so that the agency can hire more people to process classified records, handle declassification requests and appeals, and create finding aids for new records. Some of the new staff could help develop the technological capacity the archive so badly needs for managing its electronic records. Congress should earmark funding for declassification efforts within individual agencies, too. But this money should come with certain stipulations. Agencies with especially large backlogs, such as the Defense Department and the CIA, should be required to spend more on clearing them.

For years, Congress has largely abdicated its duty to oversee the declassification system. It should now step up. In 1991, Congress required the State Department to establish a historical advisory committee to oversee the production of the Foreign Relations series and to monitor the declassification and of the department’s records. Now it should make the Defense Department, the CIA, and other agencies do something similar: establish advisory panels of outside experts to review their declassification activities. Congress should further require the Defense Department to create a centralized system for reviewing declassification requests.

As part of its oversight, Congress should streamline the rules and procedures that agencies use to declassify records. It could, for example, give the National Archives’ National Declassification Center the authority to overturn agencies’ refusals or redactions of documents older than, say, 40 years. It could also empower the Interagency Security Classification Appeals Panel by requiring agencies to treat its decisions as precedents; as matters stand now, the agencies simply disregard its past rulings. For the endless digital data that agencies currently generate, Congress could mandate that an automatic declassification date be embedded in each file, so that future archivists are not buried under a mountain of classified hard drives.

Historical declassification is far from a special interest, benefitting only a coterie of obscure researchers. It should be of concern to everyone. The United States was founded on the principles of reason, openness, and accountability. Yet so long as excessive secrecy and paltry budgets keep the country’s classified records locked away, those values will be honored only in theory. And the United States will be the poorer for it.