Sen. Rand Paul continued to drive momentum for criminal justice reform Wednesday, saying the greatest barrier to jobs and voting in the U.S. is a criminal record.

“If we want to help people work and help people vote, we’ve got to fix the overcriminalization problem,” Paul said before an intimate crowd during the Bipartisan Summit on Fair Justice hosted by the Coalition for Public Safety and #cut50.

The Republican presidential hopeful advocated following California’s lead in dropping minor nonviolent felonies to misdemeanors so voting and employment opportunities are preserved for those who commit minor crimes.

Paul said the reduction also freed prison space in the state’s overcrowded system, allowing federal prosecutors to effectively detain violent felons for their entire sentences.

Overcrowding is the result of an explosion in the federal prison population since the 1980s, when politicians dropped a tsunami of harsh sentencing laws on the criminal justice system in a sweeping crime and drug crackdown.

The consequence was an 800-percent increase in the prison population over the past 30 years, with more than 208,000 people locked in federal prisons, of which half are serving for nonviolent drug offenses.

Deputy Attorney General Sally Yates said during the event that these numbers are driven largely by drug laws encompassing mandatory minimums that are not “calibrated” to match the crime to the sentence. She said this complicates the ability to distinguish punishment between a cartel leader and a low-level drug offender, eliciting steep human and financial costs.

The U.S. spends $80 billion a year on jails and prisons, a number that consumes one-third of the Department of Justice’s entire budget. Yates said this “swallows” funds that could be allocated to state and local law enforcement along with prevention and reentry programs in prisons.

“Every dollar we spend on incarcerating the nonviolent, low-level drug offenders is a dollar that we can’t spend on investigating and prosecuting the threats that we face today,” she said.

Paul noted the emergence of 10 separate bipartisan bills in the Senate seeking reform to the criminal justice system, underscoring a growing movement to push something through the usually gridlocked Congress.

But he heeded caution, reminding the audience that despite a “groundswell” in bipartisan coalitions pushing for an overhaul, legislation still faces an uphill battle.

“It’s not easy to pass legislation in Washington, and it’s not a done deal that anything’s going to pass,” he said.