On Tuesday, U.S. Attorney General Eric Holder boldly went on the public record to say that all felony disenfranchisement laws should be permanently put to bed. According to research from The Sentencing Project, one of every 13 African Americans is disenfranchised due to past felony convictions compared with one of 40 adults in general. The near-six million Americans who can’t vote due to a felony conviction is bad enough, but Holder’s reasons for repealing these laws went beyond just the present impact. Said Holder:

And however well-intentioned current advocates of felony disenfranchisement may be—the reality is that these measures are, at best, profoundly outdated. At worst, these laws, with their disparate impact on minority communities, echo policies enacted during a deeply troubled period in America’s past—a time of post-Civil War discrimination. And they have their roots in centuries-old conceptions of justice that were too often based on exclusion, animus, and fear. …

And although well over a century has passed since post-Reconstruction states used these measures to strip African Americans of their most fundamental rights, the impact of felony disenfranchisement on modern communities of color remains both disproportionate and unacceptable.

One has to wonder, though, that if Holder has found that these laws place disparate impacts on the black community then why doesn’t he just take the states that are guilty of this to court? After all, aren’t there laws that address such racial disparities?

The fact is, past court challenges to felony disenfranchisement laws have been unsuccessful. Perhaps a challenge from the nation’s top attorney might yield better results, but it’s more likely the case that our nation’s laws are too inadequate for addressing the legacy of white supremacism that’s baked into these laws.

That legacy is not a secret anymore, as evinced by the fact the U.S. Attorney General felt comfortable discussing it publicly. Over at The Daily Beast, Jamelle Bouie dug a bit more into the post-Reconstruction timeframe for disenfranchisement laws, referring to them as “legislative zombies,” lingering from a time when white lawmakers were explicitly trying to prevent African Americans from gaining political power. The New York Times editorial board weighed in as well saying the laws continue to be “guided by the ghosts of the post-Civil War era, when disenfranchisement laws were aimed at newly freed blacks.”

Marc Mauer, executive director of The Sentencing Project, made the connection between civil rights protections that are supposed to bust those Reconstruction-era ghosts (in theory) and the disparities seen today:

A half century after passage of the Voting Rights Act, nearly six million Americans are prohibited from voting due to antiquated laws that are anathema to a democratic society. The racial disparities that plague the criminal justice system are transformed into restrictions on voting that dilute the political power of communities of color.

Curiously, this is one of the few issues where there is some bipartisan agreement—and probably the only one where there is little quarrel about the racial implications. Just look where Holder made this announcement. It was not in a place traditionally considered safe to make such connections to racism, like an NAACP event, but in a room where a number of Republican lawmakers were present. One of them, Sen. Rand Paul, a GOP presidential hopeful for 2016, was in no way flummoxed by Holder’s dredging up the painful racist history behind these laws. In fact, Sen. Paul met with Holder for lunch the next day to discuss these issues further. It’s a good thing too because Paul’s own state of Kentucky has one of the strictest bans on former felons voting in the nation.

The lunch meeting indicates that Holder is hoping for legislative fixes to the racially flawed laws rather than pursuing a route through the courts. New legislation is, perhaps, the most convenient way of dealing with felony disenfranchisement as only a ruling from the U.S. Supreme Court could do away with it for good. The Civil Rights Act and Voting Rights Act are supposed to address laws that lead to disparate impacts and voting dilution among people of color. Unfortunately, right now Chief Roberts’ court is unsympathetic to helping fight racial discrimination in elections.

Speaking with Jeremy Haile, federal advocacy counsel for The Sentencing Project, he said there are three main things that trip up felony disenfranchisement cases in federal courts: One, voting is, for better or worse, the providence of states; two, the U.S. Constitution specifically carves out a ban on voting for certain criminal classifications; and three, the U.S. Supreme Court case Richardson v. Ramirez upheld that state felony disenfranchisement laws were legal for those two reasons.

That’s not to say there haven’t been attempts. Farrakhan v. Gregoire resulted in rulings that such laws violated the Voting Rights Act, but ultimately a panel of judges from the Ninth Circuit Court of Appeals overturned those decisions. Another challenge to felony disenfranchisement laws is playing out right now in Virginia. City of Mobile v. Bolden came up short in Alabama as well. Beyond that, “It’s not clear there will be an appetite soon for challenging these laws through the legal process,” says Haile.

Which might explain why Holder is working so close with Sen. Paul, and others across the aisle like Sen. Mike Lee of Utah for a legislative fix. Holder implored state lawmakers to follow suit. In Virginia, the governor’s office has already been making headway on this, along with a decades-long grassroots organizing campaign to push state legislators for a new law. Gov. Robert Bentley of Alabama also pledged support for lifting the voting ban after Holder’s announcement.

But Florida, which is the felony disenfranchisement capital of the nation with roughly a quarter of its voting-age black population impacted, has rebuffed Holder’s call, exposing a problem with the legislative solution—they don’t fully resolve the white supremacist roots of felony disenfranchisement.

Legislative fixes are fine, but until courts find the appetite for digesting the real racist impetus behind these laws, those fixes could be temporary. We saw this in Florida when former Gov. Charlie Crist began reversing felony voting ban policies only to have them reinstated when Gov. Rick Scott took office. Until courts rule these laws unconstitutional, the post-Civil War ghosts will continue to haunt.