So what does explain it? Pfaff’s theory is that it’s the prosecutors. District attorneys and their assistants have gotten a lot more aggressive in bringing felony charges. Twenty years ago they brought felony charges against about one in three arrestees. Now it’s something like two in three. That produces a lot more plea bargains and a lot more prison terms.

I asked Pfaff why prosecutors are more aggressive. He’s heard theories. Maybe they are more political and they want to show toughness to raise their profile to impress voters if they run for future office. Maybe the police are bringing stronger cases. Additionally, prosecutors are usually paid by the county but prisons by the state, so prosecutors tend not to have to worry about the financial costs of what they do.

Pfaff says there’s little evidence so far to prove any of these theories, since the prosecutorial world is largely a black box. He also points out that we have a radically decentralized array of prosecutors, with some elected and some appointed. Changing their behavior cannot be done with one quick fix.

Some politicians and activists suggest that solving this problem will be easy — just release the pot smokers and the low-level dealers. In reality, reducing mass incarceration means releasing a lot of once-violent offenders. That may be the right thing to do in individual cases, but it’s a knotty problem.

Two final points. Everybody is railing against the political establishment and experts and experienced politicians. But social problems are invariably more complex than they look. The obvious explanation for most problems is often wrong. It takes experience and craftsmanship to design policies that grapple with the true complexity of reality.

Finally, recategorizing a problem doesn’t solve it. In the 1970s, we let a lot of people out of mental institutions. Over the next decades we put a lot of people into prisons. But the share of people kept out of circulation has been strangely continuous. In the real world, crime, lack of education, mental health issues, family breakdown and economic hopelessness are all intertwined.

Changing prosecutor behavior might be a start. Lifting the spirits of inmates, as described in the outstanding Atlantic online video “Angola for Life,” can also help. But the fundamental situation won’t be altered without a comprehensive surge, unless we flood the zone with economic, familial, psychological and social repair.