At the center of the "Unite the Right" rally that turned deadly in Charlottesville last weekend was a protest of the city's plan to remove a statue of Robert E. Lee. White supremacists, neo-Nazis and others have made monuments to the Confederate commanding general a flashpoint — at times marching to keep them standing.

But Lee himself never wanted such monuments built.

"I think it wiser," the retired military leader wrote about a proposed Gettysburg memorial in 1869, "…not to keep open the sores of war but to follow the examples of those nations who endeavored to obliterate the marks of civil strife, to commit to oblivion the feelings engendered."

WATCH:The shifting history of Confederate monuments

Lee died in 1870, just five years after the Civil War ended, contributing to his rise as a romantic symbol of the "lost cause" for some white southerners.

But while he was alive, Lee stressed his belief that the country should move past the war. He swore allegiance to the Union and publicly decried southern separatism, whether militant or symbolic.

"It's often forgotten that Lee himself, after the Civil War, opposed monuments, specifically Confederate war monuments."

"It's often forgotten that Lee himself, after the Civil War, opposed monuments, specifically Confederate war monuments," said Jonathan Horn, the author of the Lee biography, "The Man Who Would Not Be Washington."

In his writings, Lee cited multiple reasons for opposing such monuments, questioning the cost of a potential Stonewall Jackson monument, for example. But underlying it all was one rationale: That the war had ended, and the South needed to move on and avoid more upheaval.

"As regards the erection of such a monument as is contemplated," Lee wrote of an 1866 proposal, "my conviction is, that however grateful it would be to the feelings of the South, the attempt in the present condition of the Country, would have the effect of retarding, instead of accelerating its accomplishment; [and] of continuing, if not adding to, the difficulties under which the Southern people labour."

The retired Confederate leader, a West Point graduate, was influenced by his knowledge of history.

"Lee believed countries that erased visible signs of civil war recovered from conflicts quicker," Horn said. "He was worried that by keeping these symbols alive, it would keep the divisions alive."

Lee himself was conflicted about the core issues of his day. He was a slave owner who some say was cruel and a general who fought to preserve the institution. But he personally described slavery as a "moral and political evil" that should end. Before the war, Lee opposed secession, but once his native Virginia voted to leave the Union he declared he was honor-bound to fight for the Confederacy.

Academics and writers vigorously debate his sentiments and strengths, but historians seem to agree on Lee's views about memorials.

"He said he was not interested in any monuments to him or – as I recollect – to the Confederacy," explained James Cobb, history professor emeritus at the University of Georgia, who has written about Lee's rise as an icon.

"I don't think that means he would have felt good about the people who fought for the Confederacy being completely forgotten," Cobb added. "But he didn't want a cult of personality for the South."

Lee advocated protection of just one form of memorial: headstones in cemeteries.

"All I think that can now be done," he wrote in 1866, "is … to protect the graves [and] mark the last resting places of those who have fallen…"

Would Lee have opposed his own monument today? Horn leaned toward yes, though he noted that it's impossible to compare Lee's views in the 1860s with the situation today.

"You think he'd come down in the camp that would say 'remove the monuments,'" Horn posited. "But you have to ask why [he would remove them]. He might just want to hide the history, to move on, rather than face these issues."