But the immediate impact will be very limited at best. First, federal inmates accounted for just 14 percent of the nation’s 1.6 million prisoners last year. Second, Mr. Holder has limited authority to enact permanent reforms without Congressional action. Third, it’s unclear how federal prosecutors will enforce his plan. To maximize its impact, the Justice Department needs to track implementation by the 93 United States attorneys around the country and hold them accountable for enforcing the policy.

For lasting national impact we need to look at the states, where most criminal defendants are sentenced. Over the past few years, a quiet revolution has been brewing in state capitals. Historically low crime rates and shrinking state coffers have led to a nascent consensus among lawmakers and advocates across the ideological spectrum that our addiction to incarceration is not sustainable, effective or humane. Republican governors in cash-strapped states have been among those leading the charge. States as varied as Texas, New York, Colorado and Michigan have passed reforms that have stabilized or significantly reduced prison populations without increasing crime.

What Mr. Holder has done is turn up the dial, lending his imprimatur to a growing sense of national urgency and moral necessity. The muted reaction to his announcement from ardent conservatives is a reflection of the shift in debate.

But this is no time to rest. Those who seek a fairer criminal justice system, unclouded by racial bias, must at a minimum demand that the government eliminate mandatory minimum sentences, which tie judges’ hands; rescind three-strikes laws, which often make no distinction between, say, armed assault and auto theft; amend “truth in sentencing” statutes, which prohibit early release for good behavior; and recalibrate drug policies, starting with decriminalization of marijuana possession and investment in substance-abuse prevention and treatment. Federal aid to state and local agencies, like the Edward Byrne Memorial Justice Assistance Grant and the Community Oriented Policing Services, must prioritize diversion and rehabilitation over arrest and incarceration.

I am not naïve about the challenge, or of the needs of crime victims. In 1992, as I was finishing high school, my 71-year-old paternal grandmother was murdered in a house robbery in Sahibabad, India. The killing remains unsolved, and the anguish it caused my family will never fade away. But in America, our criminal justice system has too often focused on vengeance and punishment (and racial suspicion) rather than on crime prevention, restitution for victims and the social and economic reintegration of released prisoners into our communities so that they do not turn to crime again.