If you take a close look at the sorry state of crime and justice in America three interwoven themes quickly become apparent. First, there are far too many people in prison or otherwise impacted by the reach of the criminal justice system. In America today, 2.3 million people are in jail or prison. The figure rises to six million if you count everyone in prison, on probation or out on parole. The vast majority of these people committed non-violent crimes. Over a quarter of our nation’s population — 65 million of us — have a criminal record.

Second, the societal costs of incarceration are staggering. Not just for the people who are incarcerated or their families, of course, but for the rest of us. The Center on Budget and Policy Priorities came out this week with a new report that reveals that 48 of the 50 states are “spending less per student on higher education than they did prior to the recession.” In Oregon, we learned last week, the state spends “about $64,000 per prisoner per year, while investing about $6,600 per university student.” And Oregon is not alone. As Pat Robertson noted last year, California also spends more on prisons than students. So do lawmakers in many other states.

Third, in far too many cases, we are mistreating our nation’s prisoners. I’ve written extensively at The Atlantic about shocking allegations of abuse and neglect of mentally ill federal inmates at ADX-Florence, the “Supermax” prison in Colorado. Last week, in Pennsylvania, disability advocates filed a lawsuit against Pennsylvania corrections officers, alleging that the state’s solitary confinement policies violate the rights of mentally ill prisoners. In California, meanwhile, a noted psychiatrist concluded that state prison officials aren’t doing enough to prevent inmate suicide. “This is a perfect example of deliberate indifference, said an attorney for an inmate who sued for help.

None of this is acceptable in a civilized nation. But virtually all of it is fixable if the nation’s political leaders exercise the will to do so. The problem of the mass incarceration of America didn’t start with a single event and it won’t end with one. There will have to be incremental reform, a gradual tilting of popular momentum toward the notion that what we’ve been doing for decades — throwing each other into prison for drug crimes — isn’t working, on any level. And, even in just the past ten days or so, there are strong signs that this is happening. I mean, the topic even made the cover of Harvard Magazine.

Too many prisoners. The federal prison budget alone has grown by nearly $2 billion in just the last five years. On Capitol Hill last week, Senators Pat Leahy (D-Vt.) and Rand Paul (R-Ky) introduced legislation that would give federal judges more discretion to avoid mandatory maximum sentences. “This bill is necessary to combat the explosion of new federal criminal laws,” Sen. Paul said, “many of which carry new mandatory minimum penalties.” Of course, the more direct solution to this problem would be to repeal or revise some of those draconian laws. And in my view the measure itself could be even better if it gave judges even more discretion. But this measure, if it passes, will ultimately result in fewer federal inmates and that’s a good thing.

The costs to society. You need only to watch Eugene Jarecki’s heartbreaking documentary, The House I Live In, to comprehend the scope and dimensions of the human toll behind these prison statistics. But there is movement. Earlier this week in West Virginia, for example, a bill that offers “community corrections options” for nonviolent offenders is moving its way toward passage. The Maryland Senate voted to decriminalize the possession of small amounts of marijuana. And then there is the New York model. “[S]ending few people into the justice system reduced mass incarceration in the entire state,” concluded James Austin and Michael Jacobson in a detailed study published earlier this year by the Vera Institute, JFA Institute and the Brennan Center.

The way we treat our prisoners. There are new signs of hope here, too. In California, a federal judge last week refused to dismiss a lawsuit challenging the state’s use of long-term solitary confinement. Officials there will have to answer substantively to the allegations made against them. In Florida, which “imprisons more youth under the age of 18 in adult prisons than any other state in the country” according to the ACLU, lawmakers were pondering a bill that would limit the use of solitary confinement upon juveniles. And in New Mexico, a man held for 22 months in solitary confinement in a county jail recently settled his lawsuit with the state for $15.5 million.

Just imagine how much cheaper it would have been for state officials in New Mexico to treat the inmate humanely to begin with. Texas alone has paid nearly $61 million in settlement costs to exonerees since 1923. And just think how all that money currently spent buttressing the private prison industry—$5.1 billion for immigration detention alone, says this piece from last summer — would help ensure that more students went to and graduated from college? The out-of-pocket costs of the incarceration of America are enormous. The opportunity costs for those millions upon millions impacted by the system — is simply too much to calculate.