WASHINGTON – Vice President Pence on Tuesday argued against pulling down Confederate monuments, calling that an attempt to “erase parts of our history just in the name of some contemporary political cause.”

“Rather than tearing down monuments that have graced our cities all across this country for years, we ought to have been building more monuments,” Pence told Fox & Friends. “We ought to be celebrating the men and women who've helped our nation move toward a more perfect union and tell the whole story of America.”

Statues honoring prominent Civil War generals and other figures have become a flashpoint across the country in light of the white supremacist rally and ensuing violence in Charlottesville, Va.

Charlottesville, Baltimore and other cities have taken down or are considering removing statutes and monuments connected with the Confederacy. Sen. Cory Booker, D-NJ, announced last week he intends to file legislation to have such statues removed from the Capitol.

President Trump has defended the statues, calling them “beautiful” and saying they reflect the country’s history and culture.

Asked by Fox’s Ainsley Earhardt if the Confederate monuments at the Capitol should be removed, Pence said “it’s important that we remember our past and build on the progress that we have made.”

While Pence appeared on Fox, and on NBC’s Today show to talk about the administration’s Afghanistan strategy Trump announced Monday, he continues to be asked to defend Trump’s controversial comments about the Charlottesville rally.

Trump claimed “both sides" were at fault for the violence, even after one alleged white nationalist was charged with murder after ramming his car into a crowd of protesters, killing 32-year-old Heather Heyer.

Trump also said there were "some fine people" among both the white nationalists and the left-wing counter protesters.

House Speaker Paul Ryan, who is close to Pence, posted a statement on Facebook Monday saying “there are no sides.”

“There is no moral relativism when it comes to neo-Nazis,” Ryan, R-Wisc. wrote.

Pence disputes that Trump made a moral equivalency. But he did not answer NBC’s Matt Lauer’s question of whether Trump’s comment about there being fine people on both sides offended him.

Noting that top GOP, business and military leaders have rebuked Trump, Lauer asked Pence if he’s putting loyalty to the president “above and in front of what’s in your heart and your gut.”

Pence replied that he knows what’s in the president’s heart.

“He condemned the hate and the bigotry that was evidenced (in Charlottesville),” Pence said. “And we’ll continue to do that.”