Is President Obama really planning a mass pardon for hundreds or even a few thousand drug offenders? The administration floated that idea on Monday in an article on Yahoo News. On Tuesday, Obama finally removed from office the U.S. pardon attorney who has served since 2008 despite sharp criticism for years that he’s mishandled his job. And on Wednesday Deputy Attorney General James M. Cole laid out new criteria for the clemency applications of nonviolent offenders.

It’s a shift that has been a long time coming. Obama has so far pardoned fewer people than any other modern president. But with tens of thousands of petitions expected, how would a mass pardon actually work? And dramatic as it may sound, would it go anywhere near far enough in reforming the pardon system, which has been a mess since the 1990s?

The president’s power of clemency, to either commute a sentence or shorten it through a pardon, comes from the Constitution. In many ways, the 18th and 19th centuries were a harsher corrections era than today, and yet early presidents pardoned convicted criminals as a matter of routine. “Before there was a federal prison system and the possibility of early release on parole, when prison sentences were mandatory and served in squalid county jails, hundreds of federal prisoners were freed by presidential fiat every year,” Margaret Love, the U.S. pardon attorney from 1990 to 1997, writes in a 2012 article.

Along the way, however, the pardon power shifted from the White House to the Department of Justice. Love explains that, charged with overseeing pardon petitions, the personal secretary for Lincoln’s attorney general decided “to keep all but the most deserving cases from coming before the kind Mr. Lincoln at all, since there was nothing harder for him to do than put aside a prisoner’s application.” In 1898, William McKinley officially routed all pardon applications to the Justice Department. Gradually, the bureaucracy gummed up the works. Franklin Roosevelt directed the DOJ to stop publishing an explanation for each pardon grant in 1933. That was a big mistake, Love points out. Without case-by-case justifications, pardoning could be seen as “ mysterious, capricious, and possibly corrupt.” Still, the pardon had a couple more moments in the sun. John F. Kennedy granted 100 pardons to first-time drug offenders serving mandatory minimum sentences. Lyndon B. Johnson continued the practice and granted 226. Jimmy Carter and Gerald Ford gave clemency to thousands of Vietnam draft evaders.

But then Ronald Reagan delegated the responsibility for making pardon recommendations from the attorney general to a career civil servant. Now the attorney general didn’t have to take responsibility for each pardoning decision, and the enterprise was shunted to the side. At the same time, pressure built from the swelling prison population. As Attorney General Eric Holder has highlighted, “since 1980, the federal prison population has grown at an astonishing rate—by almost 800 percent.” Much of the increase comes from locking up drug offenders, many of them not convicted of violent crimes, for long periods driven by mandatory minimum penalties.

All of this created the conditions for Bill Clinton’s pardon scandals. Clinton neglected the management of the pardon system throughout his time in office, Love charges, and then went on a binge in the last days of his term that made the pardon seem like a perk for the wealthy and the connected. Remember Marc Rich? When George W. Bush took over, he was cautious about using the pardon power—and problems persisted. In 2007, the pardon attorney resigned after an internal investigation into mismanagement. In 2011, ProPublica got the list of names of people who’d been denied pardons, and, in riveting work by Dafna Linzer, showed that white applicants were four times as likely as black ones to win pardons, even when they’d committed lesser or similar crimes—one more racial disparity in a criminal justice system that is rife with it.

Linzer also highlighted a particularly glaring case in which Clarence Aaron, a black man serving three life sentences for cocaine distribution who had no prior record, was turned down for a pardon even though the prosecutor and the judge supported commuting his sentence. Ronald Rogers, the pardon attorney who just lost his job, left out their recommendations in advising the White House to turn Aaron down. After another investigation, the inspector general for the Justice Department found Rogers had misrepresented the case without input from anyone more senior at the DOJ.

Obama finally pardoned Aaron last December, but the fact that Rogers stayed in office until this week is one measure of how reluctant the president has been to take on the larger breakdown in the pardoning system. Perhaps the idea was that if Obama hardly pardoned anyone, he couldn’t be accused of pardoning unfairly. Reagan and Clinton granted 1 in 100 pardon applications. Under George W. Bush, the rate fell to 1 in 1,000. In May 2012, the rate for Obama was 1 in 5,000. That’s partly because the number of applicants has risen with the prison population. But it’s also because Obama has played it so safe. Even with increasing bipartisan consensus that federal drug sentences have become too draconian and costly, the sad fact is that there’s always a political case to be made against letting people out of prison.

But now that Rogers is gone, Cole is talking not just about making more pardons, but also about addressing “older, stringent punishments that are out of line with sentences imposed under today’s laws.” He described surveying all 215,000 federal inmates as a means to identify applicants, and he has written to the 93 U.S. attorneys for recommendations. Even more promising, reform-minded groups like the ACLU, the American Bar Association, and Families Against Mandatory Minimums would play a direct role, recruiting and training lawyers to use the completed surveys to screen for eligible applicants.

Still, the pardon process is inevitably tricky and technical, and the devil is in the details. According to the Washington Post, here are the criteria for a pardon Cole laid out:

Offenders seeking clemency will have to have served at least 10 years of their sentence, have no significant criminal history, and no connection to gangs, cartels or organized crime. Applicants also must be inmates who probably would have received a “substantially lower sentence” if convicted of the same offense today. And to be eligible, they must have demonstrated good conduct in prison.

This sounds like the swing Holder took last summer at harsh sentencing, when he directed federal prosecutors to stop specifying the quantity of drugs for certain offenders so as to avoid triggering a mandatory minimum sentence. Then as now, it’s not clear how a grand pronouncement plays out on the ground. Traditionally, pardons have often turned on whether prisoners significantly cooperated with the prosecution, were seriously ill or elderly, or simply serving an unduly severe sentence. The key limiting factor on Cole’s new list of standards is the requirement that a prisoner would have done much less time if sentenced today. That could be a narrow category: drug offenders sentenced to long terms before the Fair Sentencing Act of 2010, which reduced the disparity between sentences for crack and powder cocaine.

If that’s all the proposed reform adds up to, it’s too minor a tweak to the system. It might not even address some of the most sympathetic cases—people who say they played unknowing or peripheral roles in a larger drug ring. People like Stephanie George, who said she didn’t know about the lockbox full of cocaine her boyfriend had hidden in her attic. The judge in her case 17 years ago said he had no choice but to sentence her to life, even though he saw her as “a girlfriend and bag holder and money holder but not actively involved in the drug dealing.”

In her excellent 2012 article, Love outlines a three-legged stool of comprehensive reform of the pardon power. First, the process needs clear standards, with reasoned explanations of why an application is granted. Second, whoever is in charge has to have independence from the Justice Department, real authority, and the confidence of the president. Third, the pardon process “must be accessible and responsive to people of all walks of life, and take into account the likelihood that many deserving pardon applicants will not have skilled counsel or well-connected supporters to advocate on their behalf.” In other words, the system for clemency has to be fair. For everyone—not just the prisoners sentenced under the single unduly harsh law Congress has so far seen fit to change.