“When will they storm Washington’s Monticello?” Laura Ingraham asked ominously this last week. The conservative commentator’s tweet linked to news that a statue of Jefferson Davis, one of four contested monuments in New Orleans, was slated to come down Wednesday night.

The question was met with well-deserved derision, and a subsequent correction from Ingraham — George Washington lived at Mount Vernon, Monticello was Thomas Jefferson’s estate, and neither of those Founding Fathers ever interacted with Davis, the president of the Confederacy. But it did serve to illuminate an important fact about most pro-statue protests and demonstrations: They aren’t really about the history.

Whenever it’s asked why Confederate memorials should be kept in place, a predictable raft of arguments is brought forth. “You’re trying to erase the past!” “We’re honoring the memory of valorous men!” “How will we learn from history!?”

These claims don’t stand up. The truth is that the desperation to preserve this particular “heritage” and “past” is a facade for something more malignant. It’s privileged status, not history, that’s being protected. Whether or not they’re able to acknowledge it, the thing that all these indignant commentators rush to preserve is a toxic nostalgia for the time and place that Confederate monuments represent.

[The case for the monuments in your home town: Tell us what should stay and what should go]

In most cases, as Ingraham helpfully illustrated, those protesting the monuments’ removal aren’t exactly avid historians; many couldn’t tell one Confederate general from the next. And despite what some indignant statue supporters might claim, moving memorials from the city center to a dedicated museum is nothing like leveling the Egyptian pyramids or tearing down the Roman Colosseum; it’s not destruction, it’s adding needed context. As for honoring the memories of honorable men, here’s what Robert E. Lee himself said about undue reverence for conflicts past: “I think it wisest not to keep open the sores of war, but to follow the example of those nations who endeavored to obliterate the marks of civil strife, and to commit to oblivion the feelings it engendered.”

To be clear, these monuments were raised to commemorate a time of unquestioned white supremacy, when black people “knew their place” as second-class citizens and could be punished with impunity when they stepped out of line. The Civil War was fought to preserve the antebellum South. It and the post-Reconstruction era were a time and place in which those in one segment of the population could live in the absolute certainty that they were better than the rest — that they should and always would remain on top.

That place sounds comfortable, if you’re on the winning side of things. I can understand why one might want to keep that feeling alive, especially today.

In the current moment, many Americans find themselves gripped by what some have termed “cultural anxiety.” Nearly two-thirds of white working-class Americans believe that the country’s culture and way of life have deteriorated in recent decades. Nearly half say that things have changed so much that they feel like strangers in their own country. Most hesitate to call this “racism” — after all, you don’t have to dislike people of other races to want to maintain your own status, and few see themselves as deserving of what is now viewed as the worst possible epithet.

But there’s no escaping that it’s still a question of roles, standing and hierarchy, a wish to maintain a social structure that prefers one group over the rest. It’s discomfiting to feel that you have lost power as others have gained equal footing, even if you were never entitled to dominance in the first place.

Maybe this is why the far right has taken up Old South preservation as a pet cause, and why some conservative politicians base their campaigns on their support of symbols that glorify the Confederacy (Virginia gubernatorial candidate Corey Stewart recently donned period costume to attend an “Old South Ball” and heralded the first New Orleans statue removal as evidence that “ISIS has won.”) For some, Making America Great Again means keeping alive the memory of a time when the country seemed more settled in their favor.

But there is no way to satisfy this longing for “America as it used to be” in the context of what America is becoming and should have always been: a democracy in which all citizens are of equal value. There will always be space for remembering our history, but remembering isn’t the end of it: We must also decide which parts of it are worthy of continued celebration and which are not. Those attempting to keep the past alive under the guise of protecting Confederate memorials need to recognize that this is the case, as uncomfortable as that might be.