It’s an idea so incredibly crazy it just might work: Restoring voting rights to non-violent felons—if they get back their right to own guns, too.

For some tough-on-crime conservatives, the right to bear firearms is a right that is as fundamental as the right to vote. Capitalizing on this sentiment, the strategy goes, could lead to a larger compromise on felons’ rights.

“If someone asked me if I would rather vote for mayor or have a gun, I’d rather have a gun,” said Grover Norquist, president of Americans for Tax Reform and a signatory to the conservative Right on Crime criminal justice reform coalition.

Criminal-justice reform is a hot topic in Washington, D.C. this Congress, driven by the prospect of bipartisan collaboration in an era of divided government. Leading lawmakers in both Republican and Democratic camps have proposed legislation that would address police militarization, civil asset forfeiture, and mandatory minimum sentences.

Groups such as the Brennan Center and the ACLU have also been working on reenfranchising felons in some way.

Sen. Rand Paul, the Kentucky Republican, proposed a bill last year that would restore voting rights for nonviolent felons, joining the ranks of Democrats such as Sen. Ben Cardin who believe that at least some felons should have their voting rights restored.

However, advocates of criminal justice reform are nervous about Sen. Chuck Grassley, who heads the Senate Judiciary Committee, and has not been gung ho about some of these ideas. He’s skeptical about reforms to mandatory minimums, for example, viewing them as a source of “stability in the criminal justice system.”

The thinking goes that Grassley—a senator with an “A” rating from the National Rifle Association—might be brought to the negotiating table on voting rights if the right to bear firearms were in the mix (Grassley’s office did not comment for this article).

It’s a long-shot idea, and in its embryonic stage. But tough-on-crime conservatives aren’t likely to budge on the restoration of voting rights to felons—who, they suspect, will not vote for their candidates if re-enfranchised—if they don’t get something in return.

“It is the obvious compromise,” Norquist said. “Many conservatives willing to restore voting rights would not be willing to suggest Second Amendment rights are second-class rights… In talking to conservatives, some are more or less excited about speeding up voting rights restoration. But all, when asked, agree voting rights should not precede gun rights.”

Former New York City Police Commissioner Bernard Kerik should know something about the way the criminal justice treats felons—he’s also an ex-convict.

“[Lawmakers] should give at least equal attention to voting rights, Second Amendment rights… that you are deprived of as a result of the conviction,” Kerik told The Daily Beast.

A former cop, Kerik was appointed by the Bush administration to be an interim Iraqi minister of interior following the U.S. invasion, and was also once nominated to be U.S. Secretary of Homeland Security. He withdrew his nomination after he acknowledged failing to pay taxes for a nanny he hired. After pleading guilty to charges relating to this tax issue, he was sentenced to several years in federal prison.

The theft of oysters or harvesting too many fish commercially can make you a felon, Kerik said. And, as he too well knows, so can a federal tax charge.

“I possessed a firearm for this country for 35 years. I've used a firearm personally when my partner was shot in a gun battle… I was convicted of false statements on tax charges primarily relating to my children's nanny, but I can never possess firearm again for the rest of my life. Is it fair? No.” Kerik told The Daily Beast.

Kerik is also planning to launch a nonprofit organization to press for criminal justice reform in the next several weeks.

Among libertarians working on the criminal justice issue, there is some initial support for the idea, even in its early stages.

“Obviously, we’d need to see details of any proposal, but we’d be very likely to support a bill that restored voting and Second Amendment rights to nonviolent offenders who made youthful mistakes,” said David Pasch, spokesman for Generation Opportunity, a Koch-backed youth advocacy group.

Clark Neily, a senior attorney at the libertarian Institute for Justice, said he has heard about the prospect of combining voting and Second Amendment in a broader effort to restore rights to some felons. He approves of rights restoration broadly, but disapproves of the idea of a political trade on the issues.

“If what is going on is trying to limit the extent to which people are dispossessed of political rights, great. But if it’s a political ploy, I find it distasteful,” he said. “If it is in fact a trade-off, I don’t like the idea of horse-trading when it comes to liberties, or constitutional rights.”

Much of the momentum for criminal justice reform on the right has been created due to renewed efforts by libertarians like the Koch brothers. However, many of the major groups operating in this policy area—such as the Charles Koch Institute, the Institute for Justice nor the Right on Crime coalition—have yet to take a formal stance on the restoration of Second Amendment rights to nonviolent felons.

Under federal law, felons lose their right to bear firearms, unless their rights are individually restored by a federal agency or through litigation. Felons are subject to the laws of their state when it comes to their right to vote after their time is served. In 11 states, felons lose their right to vote forever, while in two states felons continue to have the right to vote even while in prison. The remainder of the states have some sort of limitation on voting rights for felons.

For now, as the idea is being mulled, the legislative prospects for the trade-off are not good. If any compromise is made on the issue, it will likely be first formed off of Capitol Hill by outside criminal justice reform groups, away from the political poison pill of restoring rights to felons, even nonviolent ones.

“Tons of momentum in the public for criminal justice reform, but not nearly as much in the Republican caucus,” said a top Senate aide who works on the issue. “Many of the Republican caucus were elected when tough-on-crime was a driving force.”

Prison reform, civil asset forfeiture reform, and a juvenile justice bill are far more likely to pass in the current political environment, the aide said.

But Norquist argued that if progressive lawmakers are serious about helping felons rejoin society, the restoration of firearms rights should be on the table.

“If someone thinks [ex-felons] should not be trusted with a gun, why would you trust him with voting for the government, which is the legal monopoly on force?” he said.