When Major Spencer Williams was ordered to “shut down shop and move out” of Afghanistan in 2005, he closed his final message from the field as he always did—quoting a long-dead historian. “Plant yourself not in Europe but in Iraq; it will become evident that half of the roads of the Old World lead to Aleppo, and half to Bagram.”

Williams made up one-third of the U.S. Army’s historical field staff in Afghanistan—a team directed to cover the breadth of the country, vacuuming up media, documents, and oral histories so that some future soldier or academic could better understand the course of the war and how one might respond to circumstances should they arise again. The war offered more than enough material to keep Williams and the others busy, but they weren’t able to communicate the importance of that work to those leading the mission in the country. Following a command from the highest-ranking officer in Afghanistan, the historians were on their way out of Kabul.

It would be almost two years until another team came back into the country. In that time, units cycled in and out of the war zone, each adding a small drop to the bucket of the longest U.S. military engagement in history. Whenever a unit prepared for the return trip home, its soldiers collected their gear, prepared the site for the following unit, wiped local servers, and allowed the details of the prior months to fade.

The gap in record-keeping created by the absence of Williams and his team—and the difficulties they faced in demonstrating this record’s value while overseas—illustrate a common headache of 21st century historians. Though technology has made more sources than ever available to color, verify, and explore history, determining those sources’ value remains the task of a trained human eye. And in the case of the Army, support for that eye has declined as its necessity increased.

Williams and his team were sent to Afghanistan in order to collect material for inclusion in the Army’s Tan Books, histories of the wars in Iraq and Afghanistan written by the Center of Military History. CMH’s official histories, which have existed in various forms since the years following World War I, trace narratives of wars using material only a massive enterprise like the Army can synthesize. They offer maps of documents flying through the Pentagon and overseas bases everyday, and examine points in which the arc of history bends ever so slightly. The sprawling nature of the histories also means that precise connections between events can be found and applied.

We did a disservice to the American public.

Jill S. Russell, a visiting professor of national security and strategy at the U.S. Army War College, recalls digging through niche volumes of CMH’s The United States Army in World War II, also known as the Green Books. “There were things that guys coming back from the Philippines were writing that are being repeated by guys coming back from Afghanistan almost literally word-for-word,” she says. Russell cited modifications made by soldiers in World War II to improve the ease of use of heating elements in mountainous terrain. Upon deploying to similar terrain in Afghanistan, some soldiers rediscovered the modifications made by their predecessors.

Modern wars have upended some of the most basic factors CMH relied upon in order to write histories in a consistent way. “Since 1991, the operations we’ve gotten into—Somalia, Haiti, Bosnia, Kosovo—were more complicated scenarios,” says Shane Story, a retired Army helicopter pilot who’s overseeing the creation of the 30-some eventual volumes of the Tan Books. “Even more since 9/11, both Afghanistan and Iraq, they’re not clean, neat narratives. ... [In] the best work that you write, you know what the last word is before you start. We don’t know what the last word is.”

Story and Jerry Brooks, who’s responsible for collections in the field, speak candidly of the problems CMH historians face as they work through the Tan Books creation, a process that will take decades to complete. Once, there were scores of clerks responsible for maintaining detailed records of units. With the rise of computers and software, the Army believed that “everybody would be their own records manager,” and broadened this responsibility to average soldiers, Brooks says. “Well, they failed to take into consideration that people are lazy.”

In theory, these individual record-keepers hand off documents and other sources of information to field historians when requested. However, in addition to the two years after Spencer Williams and his team left Afghanistan a decade ago, the country hasn’t had a field historian from the Army since 2014. Historians dealing with Iraq have avoided these same gaps, but still suffer from fewer personnel in the field than in past wars. Brooks cites Vietnam, where U.S. headquarters in Saigon alone maintained a staff of more than 20 historians. Nowadays, as a result of caps on the number of troops deployed, historians oftentimes find themselves on the first flights back home.

“We did a disservice to the American public, because we did not put enough historians downrange to collect the documents. And now we're reaping what we’ve sowed,” Brooks says.

By his estimate, more records from the wars in Iraq and Afghanistan have been wiped or otherwise lost than remain in existence. But he admits the exact figure is difficult to determine due to the few historians in the field. A concept document that lays out the plan for the Tan Books supports this estimate, listing the amount of data lost in the wars as “incalculable.”

Both Brooks and the Tan Books concept document exclude the multitude of informal records created by soldiers during the rise of the digital age—emails, Facebook posts, and blogs, to only name a few. These materials are both an opportunity and a source of unanswered methodological questions, according to Michael Gisick, a Ph.D. candidate at Australian National University who studies the use of social media by U.S. service members during the war on terror. “I suspect there are great quantities of images, emails, and other digital narratives tucked away on the hard drives of America’s veterans,” he says, explaining that these sources of information can “jog the memories of participants, spur questions, and illustrate events,” as well as counter popular narratives that might have arisen in political circles and at higher levels of the military.

A good historian “can build the story around just one document,” if necessary. But finding those capable historians is still a challenge.

Even for historical organizations that have been quick to adopt online technology, the role these sources can play is still being tested and explored, says Russell Riley, who co-chairs the Presidential Oral History Program at the University of Virginia’s Miller Center. The program he helps direct, whose texts are of a similar scale to the Army’s official histories, is examining how to use social media as an additional springboard for the Obama administration histories, and he expects they’ll play an even larger role once they begin considering the Trump presidency. He believes that as officials of all stripes have become more reluctant to put thoughts down in writing, social media can offer a less filtered means of communication.

CMH notes that more than 150 terabytes of data have been collected for cataloging and eventual use in the Tan Books. The value of that data remains to be seen. Detachments sent into the field by CMH, though trained to recognize historically significant documents and conduct interviews with people of interest, act as “vacuum cleaners,” as Brooks has called them, rather than analysts. When Brooks briefed a group of Army historians, he included a photo of a Burger King fry holder tagged as coming from a restaurant set up after the invasion. “Technically Correct but Useless” reads the title—a classification that might only increase by including the Facebook posts of young, deployed soldiers.

Despite these limitations, Story remains convinced that the “glass is half-full” for the Tan Books. Like the famed Civil War historian and author Shelby Foote, he believes a good historian “can build the story around just one document,” if necessary. But finding those capable historians is still a challenge. Right now, Story can dedicate one individual to work on an actual text of the Tan Books at any one time. He’s pleased with the work already released, a 70-page pamphlet on the 2007 to 2008 surge of American troops in Iraq, but notes that a complete text stemming from the work is still six or seven years down the line from release.

“In certain respects, I think of ourselves as having a role comparable to the government accounting office,” Story says, arguing that even first drafts of the texts play an important role. Russell and Brooks agree, saying that these early efforts, which few expect to be complete or even immutable, will drive further research and questions as more becomes known and unclassified about the wars.

Even the best histories don’t provide a means of knowing everything ahead of time, Story says. But “when circumstances arise, done well, at least, I think it can give you a means of judging them with a little more acuity.”