A new history tells the story of four Union agents. One tale, little known, is as compelling as any Hollywood epic

Douglas Waller had a confession to make. “I’m a little intimidated here,” he told an audience that included archivists and historians at the National Archives in Washington. “I’m not a civil war historian. I don’t even play one on TV.”

Ken Burns on America: 'We're a strange and complicated people' Read more

Waller, 70, used to cover the CIA for Newsweek and Time and has written biographies of senior intelligence figures in the second world war. So in writing Lincoln’s Spies, an “ensemble biography” of four secret agents during the American civil war, he brings a fresh eye to a timeworn subject.

“When we think of the civil war,” he said, “we have this image of Union and Confederate soldiers posing stiffly in Mathew Brady photographs, or the soft violin music of a Ken Burns documentary which always put me to sleep, and mangled bodies strewn on desolate battlefields where thousands died in mass assaults like Pickett’s Charge. And all that happened.”

But what also happened between 1861 and 1865 was a revolution in how armies spied on each other. Conditions were perfect for going undercover: each side spoke the other’s language and knew their culture and customs.

“If you were a northern spy, it wasn’t too difficult to fake a southern accent,” Waller said.

He tells his story through three men and a woman who spied for the Union: Scottish-born Allan Pinkerton, George Sharpe, Lafayette Baker (a “publicity hog and bureaucratic bigfoot”) and Elizabeth Van Lew.

“I found Union operatives to be far more interesting than their Confederate counterparts,” he said. “Two of my spies were heroes in this war, one was a failure and the other was a scoundrel, so I had a pretty good mix of characters.”

Van Lew is arguably the star. While other southern belles were baking cakes for their menfolk, sewing their uniforms and urging them off to war, she was risking all by acting as the Union’s eyes and ears in Richmond, Virginia, the capital of the Confederacy. So compelling is her story that a member of the audience at the archives suggested it would make “a great movie”.

The daughter of a wealthy merchant and a highly educated socialite, Van Lew developed empathy for enslaved people she saw being beaten in the streets. She was sent to Philadelphia to gain an education; her governess gave her a revulsion towards slavery and commitment to abolition.

Facebook Twitter Pinterest Elizabeth Van Lew. Photograph: The Granger Collection/Alamy

When her father died in 1843, Van Lew spent much of her inheritance helping enslaved people flee north or secretly paying wages to African Americans who stayed at her mansion.

Waller said: “She was a short woman who had been quite beautiful in her youth but, when the civil war started, she was in her 40s and unmarried and she was considered by Richmond society to be an old maid.

Van Lew was described as brilliant, clever and “acid-tongued”. When Union prisoners poured into Richmond, her conscience would not allow her to stand idle. She persuaded the authorities to let her bring books and meals and to tend the wounded. That made her a pariah.

Her neighbours underestimated her. Van Lew built a spy network including farmers, factory workers and African American servants that produced three reports a week on topics including the city’s defences, troop movements and local morale. Messages to Ulysses S Grant, sent in code or invisible ink, were accompanied by a rose picked from her garden.

Waller said: “Van Lew’s Church Hill mansion became her spy ring’s base station. She had several dozen agents and couriers working for her. Each one carried a carved peach seed that identified them as a member of her network.”

She was hiding in plain sight. There was a lengthy investigation, with her sister-in-law testifying against her, but the rebels wrote her off as just a harmless woman who talked too much.

“Basically,” Waller said, “she was saved by southern sexism.”

‘She felt abandoned’

Waller spoke further in an interview. “Interestingly, she never considered herself a spy,” he said. “She considered herself a loyal patriotic American who was helping America, not spying on the Confederates. She considered Confederates and the Confederate government to be disloyal.”

The point was illustrated when Union forces took Richmond.

“It was total anarchy and Elizabeth Van Lew had had a Union general secretly send her an American flag and she went up to the top of her mansion and hoisted that flag up. Soon a huge crowd surrounded her mansion, threatening to burn it down and practically lynch her. She looked at them all with that stare and said: ‘You touch one board on this mansion and I’m gonna set the Union army on you. You’re going to pay for it.’ She scared them off.”

The south did not greet Union forces as liberators. Van Lew remained isolated. Waller said: “I’ve always said it’s easy to be a liberal in Vermont and a conservative in Alabama. It’s hard to be a conservative in Vermont and a liberal in Alabama. Well, she was a liberal in Richmond even after the war. She was very upset that the old power structure basically came in.”

Van Lew lived to see a monument to Robert E Lee erected, cheered by thousands. “She was just totally despondent. It was basically a not too subtle message that the Confederate army may have lost but the people who started the secession and created this disaster are back in charge now and will be in charge for a long time. She and other Unionists were very depressed by those monuments.”

Facebook Twitter Pinterest Allan Pinkerton, left, with Abraham Lincoln and Maj Gen John McClernand at Antietam in October 1862. Photograph: Alexander Gardner/AP

Grant appointed Van Lew postmaster of Richmond but she lost the job after his presidency. Deeply unpopular, she died in 1900 nearly broke and was buried vertically in an unmarked grave. A granite headstone was added later.

Waller added: “She was deeply resentful. She and other members of her ring felt abandoned, that they’d sacrificed their personal safety, their fortunes, and didn’t end up with much. It was pretty sad for her.”

Lincoln’s Spies also traces the impact of technology on civil war espionage. Photography became a vital tool. The telegraph made signals intelligence important Union forces, in particular, used aerial reconnaissance: hydrogen-filled balloons were as sent as high as 1,000ft, aeronauts dangling in wicker baskets to scope out battlefields.

By the end, Grant had a better idea of the count of Lee’s forces than Lee himself had. Lee’s old-fashioned views were his undoing.

“It was, in some ways, a different culture,” Waller said. “Lee didn’t trust spies. They weren’t engaged in ‘chivalrous war’ so he didn’t think too highly of them. He didn’t think too highly of women spies and didn’t think women were capable of taking intelligence. He thought most spies were amateurs and what they produced wasn’t a bunch of value.”

‘It wasn’t breaking news that Lincoln was in danger’

The president embraced the more modern approach.

“Lincoln knew how to keep a secret,” Waller says, “and how to operate in secret and he could be ruthless when he when felt he had to be, suspending the writ of habeas corpus, jailing thousands of people and closing down newspapers hostile to his efforts and at one point threatening to disband state legislatures that were going against him.”

So how would Lincoln feel about today’s CIA black sites? “The popular image is ‘Honest Abe’ would be revolted by it. No. Not necessarily.”

She was a liberal in Richmond even after the war. She was very upset that the old power structure basically came in Douglas Waller on Elizabeth van Drew

The book took nearly five years to write and has been hailed as a major contribution to civil war literature. But Waller steers clear of grandiose claims. “Espionage didn’t win the civil war,” he said, frankly. “The north had an overwhelming advantage in population and resources and basically wore down the south.

“Now, did espionage play a part? Of course, and there were instances on the battlefield where it helped out. There were also instances where it didn’t help out. Even if the south had the best intelligence apparatus and the north had the worst, which was the exact opposite of the case, the north still would have won.”

The limitations of espionage were made plain in the immediate aftermath of the war, when John Wilkes Booth slipped into Ford’s Theatre in Washington and assassinated Lincoln. As intelligence failures go, it must rank with the killing of John F Kennedy and the 9/11 terrorist attacks.

Waller said: “It wasn’t breaking news that Lincoln was in danger of being assassinated. He was in danger of being assassinated before he got elected as a candidate. He kept his own file of hate mails and threats. In fact, some people were amazed that he survived this long in the presidency considering the hostility toward him in the south, where people would run ads in the newspaper offering rewards literally for his head, and in the north where there was a lot of pro-Confederate sentiment or criticism of the ways he conducted the war.

“He had very little protection to speak of and didn’t want a Praetorian guard around him. He thought that kept him from the people. It’s amazing he stayed alive as long as he did.”