In 1831, Alexis de Tocqueville visited the Eastern State Penitentiary in Philadelphia to observe first-hand the effects of a peculiar — and, at the time, entirely novel — form of incarceration. The Quakers, who had opened the prison two years earlier, believed that long-term solitary confinement was an ideal form of religious penitence (whence the termpenitentiary) and would hasten prisoners’ rehabilitation and reintegration into society. They saw it not as extreme punishment but as a progressive idea, far preferable to the giant holding pens typical of the age, where mutilations and violence among prisoners were common, and spiritual betterment all but unthinkable.

Tocqueville was favorably impressed. “Can there be a combination more powerful for reformation,” he wrote, “than that of a prison which hands over the prisoner to all the trials of solitude, leads him through reflection to remorse, through religion to hope, and makes him industrious by the burden of idleness?"

Ten years later, Dickens paid his own visit to Eastern State, and came away with a rather different opinion. Solitary confinement, he found, inflicted unimaginable torment on the minds of those subjected to it. Far from leading prisoners to enlightenment, it ruined their concentration and haunted them with hideous visions. They fell into deep despair, losing track of time and of themselves. “I hold this slow and daily tampering with the mysteries of the brain, to be immeasurably worse than any torture of the body,” he wrote in his American Notes for General Circulation:

[B]ecause its ghastly signs and tokens are not so palpable to the eye and sense of touch as scars upon the flesh; because its wounds are not upon the surface, and it extorts few cries that human ears can hear […] It wears the mind into a morbid state, which renders it unfit for the rough contact and busy action of the world.

Dickens was not alone. Harry Hawser, who wrote poems about his experience at Eastern State around the same time, hauntingly described the effects of being plunged into a “living tomb.” By the end of the 19th century, the Supreme Court noted that solitary confinement had caused many prisoners to fall “into a semi-fatuous condition,” and others still to kill themselves or to become violently insane. By World War I, the practice was largely abandoned.

Still, the idea never entirely went away, and in our bewildering world of chronically overcrowded, gang-infested prisons, it has returned with a vengeance. The new generation of high-security supermax prisons, whose spreading popularity over the past 40 years has coincided with an explosion in prisoner numbers, is premised on the notion that dangerous inmates — the “worst of the worst,” in official parlance — need to be kept separate from the general prison population, and from each other.

The term “solitary confinement” is scrupulously avoided in favor of other, more clinically bureaucratic terms like “security housing unit” (SHU) and “administrative segregation.” But the result is essentially the same as it was in 1840s Philadelphia. Prisoners are deprived of almost all human contact — not just for days or weeks but in many cases for years on end. They spend 22 or 23 hours a day in cramped cells without windows, with no work or other structured activity, and with limited access to books, television, and other outside stimulation. If they talk to their fellow prisoners at all, it is by shouting through the plumbing system. Visits are difficult or impossible to arrange, and contact with the outside world — even the opportunity to see or send photographs — is rare to nonexistent. Their only experience of touching another human is when they put their hands through a slot in their cell doors to be cuffed or chained en route to their hour-long daily exercise in an enclosed concrete pen.

Tocqueville and the Quaker model are gone, but solitary confinement retains its share of supporters. Jeffrey Beard, the head of the California Department of Corrections and Rehabilitation (CDCR), claims that the authorities are in a state of virtual warfare with vicious criminals bent on fomenting “terror,” more or less for the sake of it. Keeping prisoners isolated, he and his fellow prison officials argue, is the only way to keep mayhem and violence at bay.

On the other side are prisoners, their advocates, and human rights organizations, who argue that long-term solitary confinement falls under international definitions of torture. It does not diminish prison violence, they say; it can only exacerbate unrest. Too many prisoners are moved into isolation cells for indefinite periods without due process and without a justifiable reason. In California, which keeps about 12,000 inmates in long-term isolation (out of 80,000 nationwide), prisoners do not have to be proven gang members to qualify; mere “association” is enough. And the decision on what constitutes association is left to the whim of prison administrators meeting behind closed doors. Neither the inmates themselves nor their lawyers have any say, or any right to challenge the evidence. A number of documented cases demonstrate the shocking paucity of some of that evidence: it may be no more than a choice of reading material, or the coincidence of having been placed in a cell with a known gang member, or something as innocuous as a coincidental symbol of a Christmas card.

Once inmates have been tarred, rightly or wrongly, with the gang label, they are liable to be thrown into isolation, with their parole eligibility suspended indefinitely. Whatever their original crimes, they now serve a de facto life sentence.

The battle lines over solitary confinement are now more than ever entrenched, and the reasons justifying the practice have become significantly more retributive. The problem is not just a failure to learn the lessons of the past. It is also that, when it comes to prisons, people tend to see only what they want to see.

***

In early July, 30,000 prisoners across California began what became a 60-day hunger strike to demand a major overhaul, if not outright abolition, of the solitary confinement regimen. They also made demands for better food, better access to medical care, more human contact, fresh air, and sunlight. In so doing, they attracted more media attention — and public sympathy — to their plight than at any time since the opening of California’s most notorious supermax prison at Pelican Bay, in the far north of the state, in 1989. The new protest was significantly larger than two earlier hunger strikes in 2011, which had been premised on the same idea: that something dramatic had to be done to draw the attention of an otherwise apathetic public to conditions unworthy of a democratic society, and to move otherwise intractable public officials to corrective action.

The causes of such widespread prisoner unrest are not hard to discern. California not only keeps more prisoners in solitary than any other US state, it also treats them more harshly — because so many are gang “associates” rather than full members — and holds them for longer. In the words of a class action lawsuit working its way through the federal courts, the state is an “outlier in this country and in the civilized world.” At Pelican Bay alone, about 500 prisoners have been in solitary for more than 10 years. (The prison houses about 3,000 inmates, one-third of them in “security housing.”) About 90 have been in isolation for more than 20 years, and one prisoner has been on his own for 43 years and counting.

Usually, the only way out of solitary is to “debrief” — in other words, for a prisoner to tell the authorities everything he knows about his gang connections. But debriefing is fraught with problems. Bona fide gang members face all kinds of risks from their fellow prisoners once they turn snitch, so they tend to keep quiet. And many of those labeled gang “associates” — 78 percent of the total at Pelican Bay, according to official records from 2011 — have little or no information to offer in the first place.

Take Gabriel Reyes, one of the class action plaintiffs, who was originally convicted of housebreaking and sentenced under California’s draconian three strikes law. He was thrown into solitary 17 years ago, based on the mere fact that he was seen exercising with known gang members. Since then, everything from his tattoos to his Mexican-themed artwork has been held against him as evidence of gang association. He has not hugged his daughters in two decades, since they were in preschool, a startling deprivation for a man without a violent record. Some time ago, Reyes described to his lawyers how he and his fellow inmates were ready to “explode.”

After the 2011 strikes, authorities promised a number of reforms but delivered on only a handful. Prisoners are now allowed to hang a calendar in their cells where previously they had to keep their walls bare, and they are given a handball for their daily exercise in the concrete yard known as the “dog run.” About 250 prisoners in solitary — roughly two percent of the total in California — have also been deemed eligible for return to the general population, although it is not clear how many of them have actually been transferred. Other demands have gone unheeded — even relatively modest ones like a weekly phone call, or an annual photograph to send to family and friends. A 2012 document outlining the state’s proposed changes to its system of gang “validation” (identifying inmates as members or associates) has been condemned by lawyers representing some of the most prominent Pelican Bay inmates as “making virtually no meaningful changes and, instead, expanding the net of who may be incarcerated in the SHU [Security Housing Unit].”

This summer’s strike began with an extraordinary sense of purpose. Leaders of the most feared gangs — black, Mexican and neo-Nazi — signed a pact setting aside their mutual animosity to unite instead against their common enemy, the Department of Corrections and Rehabilitation. They managed to spread word of the strike across a dozen high-security prisons. While the number of prisoners refusing to eat inevitably tailed off over time, the sheer determination of those left at the end took prison officials by surprise. “A couple of these guys are really serious,” prisoners’ rights lawyer Charles Carbone told me a few weeks in. “Whether or not they are in a gang, a lot of them have a soldier mentality. They believe one or two soldiers need to be sacrificed for a greater purpose, and they are willing to do it. They have very little to lose.”

Until the final phase of the hunger strike, California officials took their own hard line in response. “Don't be fooled,” Beard wrote in a strikingly aggressive op-ed piece in the Los Angeles Times in early August:

Many of those participating in the hunger strike are under extreme pressure to do so from violent prison gangs, which called the strike in an attempt to restore their ability to terrorize fellow prisoners, prison staff and communities throughout California […] We're talking about convicted murderers who are putting lives at risk to advance their own agenda of violence.

Beard and the CDCR, in other words, took the view that making concessions during the previous strike had been a mistake. And they did their best to crack down even harder on a group of prisoners with precious little in the way of privileges left to take away. According to prisoners’ advocates, certain attorneys were told they could no longer talk to their clients. Exercise time for the hunger strikers was reduced, prisoners had property confiscated, sandbags were placed at the bottom of their cell doors, and contact with the outside world — particularly mail from their families — was restricted or cut off altogether. In late August, several dozen Pelican Bay inmates were woken in the middle of the night and dispersed around the state to isolate them even further.

The state made clear it was willing to force-feed prisoners to keep them alive, even if they had previously signed do-not-resuscitate orders. The prospect brought back memories of force-feeding at Guantanamo Bay — a practice condemned by the United Nations Human Rights Commission as a form of torture. But the State argued in federal court, just as Beard did in the Los Angeles Times, that many of the hunger strikers were being coerced by prison gang leaders. It was an argument undermined each day by dozens, sometimes hundreds of prisoners, who gave up on the hunger strike with no sign of retaliation or reprisals.

In the end, both sides showed signs of compromise. The warden of Calipatria State Prison, between the Salton Sea and the Mexican border, persuaded his 22 hunger strikers to resume eating in exchange for extra television channels, better food in the canteen, and permission to make monthly phone calls beginning this fall. Tom Hayden, the veteran activist and politician who has successfully negotiated truces between Los Angeles street gangs, pressed the idea that similar accommodations could end the strike statewide. When that initiative failed, he approached two state legislators and persuaded them to hold hearings on the conditions in California prisons, including solitary confinement, starting later this month. The remaining strikers, some of them getting close to the end, took this breakthrough as a form of victory and agreed to resume eating.

***

The key issue for the legislative hearings is the fact that prisoners in solitary have effectively been placed in a special legal category, where they are denied rights simply because of who they are — or, more precisely, who the government says they are. Some may indeed be the “worst of the worst,” while others, like Gabriel Reyes, do not appear to belong in a high-security unit at all. Whatever their original crimes, they are now punished without recourse to courts, without legal review, and with little or no prospect of reversal. It’s a “scorched earth solution,” says Carbone. “The attitude is, we don’t care if we’ve got all the right people or all the wrong people; we just know we’ve got some of the right people.”

Just as the Bush administration relied, in the wake of 9/11, on public tolerance or even support for the treatment of detainees at Guantanamo, the California government appears to be counting on the indifference or outright hostility of its own population toward high-security prisoners. Even avowed liberals like Jerry Brown, California’s governor, and Kamala Harris, the state attorney general, seem to see nothing wrong with the system and are fully behind Jeffrey Beard’s hardline stance. In a response to the class action lawsuit, Harris said that long-term solitary confinement did not qualify as cruel and unusual punishment under the Eighth Amendment because it did not constitute “a sufficiently serious injury” and did not meet the level of “deliberate indifference” on the part of state officials.

That’s an argument contradicted by a vast and all but unanimous penological literature condemning solitary confinement as a punishment unfit for civilized society. Dickens’s observations at the Eastern State Penitentiary have been amply borne out by more recent medical studies. “Between a quarter and a half of the prisoners in the SHU’s I have visited or read about suffer serious and long-term mental illnesses,” the Harvard psychiatrist Stuart Grassian reported back in 1998. “They are, on average, the most severely psychotic people I have seen in my entire 25 years of psychiatric practice.

Symptoms reported by Grassian and many others, include massive free-floating anxiety, an inability to concentrate or remember anything, distortions and hallucinations, hyper-responsiveness to the slightest noise or other external stimulus, and a tendency to lash out in sudden destructive, often self-mutilating outbursts. Prisoners going into solitary sometimes imagine they can take advantage of their isolation to read, or study, or develop an interest in painting, but, invariably, they grow listless and unfocused within just a few days — unable to concentrate for even short periods of time. In a 2003 paper, Craig Haney of the University of California, Santa Cruz noted: “There is not a single published study of solitary or supermax-like confinement in which nonvoluntary confinement lasting for longer than 10 days […] failed to result in negative psychological effects.”

The evidence of these studies clearly contradicts the official line that isolating prisoners is a necessary measure to reduce prison violence. Violent incidents at California prisons have actually increased by almost 20 percent since Pelican Bay opened and long-term isolation became institutionalized statewide. When a national commission spearheaded by a retired federal appeals judge and a former US attorney general looked into the matter in 2006, they concluded that responsibility for prison violence lay primarily with the prison authorities, not the prisoners themselves. A system that either packs prisoners into overcrowded cells or isolates them, then fails to provide an adequate daily structure of work, exercise, reading and socializing, is a system ready to explode. “Prisons don’t have to be as dangerous and violent as they are,” the director of California’s Prison Law Office, Donald Specter, told the commission. “The culture of our prisons virtually dictates the level of violence that you will have in them. And if you change that culture, you will reduce the violence.”

There appears to be only a tenuous connection, if any, between the nature of an inmate’s crimes and the violence he or she exhibits once inside. As Grassian, the Harvard psychiatrist, wrote in 2006:

As one passes through the levels of incarceration — from the minimum to the moderate to the maximum security institutions, and then to the solitary confinement section of these institutions — one does not pass deeper and deeper into a subpopulation of the most ruthlessly calculating criminals. Instead, ironically and tragically, one comes full circle back to those who are emotionally fragile and, often, severely mentally ill.

International law is unambiguous on the subject. In 2011, the UN’s Human Rights Council put out a report that declared solitary confinement for more than 15 days unjustifiable under any circumstance. If it is used for punishment — something the state of California insists it does not do, despite ample indications to the contrary — it violates the international Convention against Torture (of which the United States is a signatory), because “it imposes severe mental pain and suffering beyond any reasonable retribution for criminal behavior.” The author of that UN report, Special Rapporteur Juan Mendez of Argentina, reiterated his conclusions in August in a statement seemingly directed at California. “I urge the US government to adopt concrete measures to eliminate the use of prolonged or indefinite solitary confinement under all circumstances,” he said.

This is not a fight that California officials seem destined to win. Indeed, one of the lessons to emerge from the hunger strike is that the more trenchantly they defend solitary confinement, the more the public comes to understand it as a moral blight and an international embarrassment. That was certainly evident from the Los Angeles Times’s mailbag. At first, the paper noted, a majority of readers wrote in with a “let them starve” mentality, but as the strike wore on, their positions quickly shifted the other way.

Even the Justice Department has challenged Brown, Harris, Beard and their allies, issuing a brief in early August expressing concern that keeping mentally ill prisoners in solitary confinement posed a risk of “serious harm.” Interestingly, the opinion was based largely on a DOJ investigation of solitary confinement in Pennsylvania, where Jeffrey Beard was head of corrections before he moved to California.

***

The most common explanation of how American prisons came to rely so heavily on long-term solitary confinement has to do with the explosion in the national prison population since the 1970s. For a variety of reasons — the return of the death penalty, the introduction of mandatory sentencing laws, the rise of inner-city street gangs, the “War on Drugs,” and a cycle of political influence-peddling and tough-on-crime legislation sometimes referred to as the prison-industrial complex — inmate numbers increased fivefold over the last 25 years of the 20th century.

The jump was not a product of some unprecedented crime wave; in fact, violent crime in the US has been declining steadily since the 1980s. But it did create a crisis inside the country’s prisons, which were not equipped to absorb such numbers, and could not keep discipline over their chronically overcrowded corridors. Prison gangs formed, first as a form of protection for one group of inmates against another, and then as a counterforce to the guards who ended up ceding significant authority to them.

Matters came to a head when two guards at a prison in Marion, Illinois, were killed on the same day in 1983. The warden decided to introduce a system of permanent lockdown for the most dangerous inmates, keeping them in isolated units. And so the concept of the SHU was born.

Pelican Bay came along six years later, followed by supermax prisoners in more than 30 states —many with isolation units. California now has four SHUs — at Corcoran, Folsom, and Tehachapi, as well as Pelican Bay. Long-term solitary confinement was seen as the quickest possible fix to a prison crisis spiraling out of control. And when it didn’t work — when the violence and gang activity either increased or failed to subside — the response was to expand solitary confinement even more.

Not everyone sees the problem strictly in terms of crisis management. The growing use of solitary confinement has also coincided with radical shifts in judicial thinking concerning the rights of prisoners, and a growing racial imbalance in the prison population, leading the civil rights advocate and legal scholar Michelle Alexander to dub this era the “new Jim Crow.” Just as the black revolutionary movement did in the 1960s, many see the mass incarceration of African American men as an explicitly political attempt to deny them their constitutional rights.

It’s a view that judges and prison officials, intriguingly, have been known to share — and not necessarily limit to African American inmates. In 1975, before the current era of long-term solitary confinement, the warden of the Marion penitentiary, Ralph Aron, said in federal court: “The purpose of the Marion Control Unit is to control revolutionary atti­tudes in the prison system and in society at large.” And the judge in the case agreed. The control unit, he said, “has been used to silence prison critics. It has been used to silence religious leaders. It has been used to silence economic and philosophical dissidents.”

In 1961, a national conference of prison wardens and administrators gave a warm reception to an MIT psychology professor named Edgar Schein, who had studied the effects of brainwashing on US soldiers held captive by North Korean and Chinese communists and believed similar techniques, including the isolation of prisoners and the segregation of leaders from their followers, could be adopted back home to counter the nascent Black Power movement. After hearing Schein’s presentation, Bertram Brown of the National Institute of Mental Health told the conference: “Undertake a little experiment of what you can do with [black] Muslims […] Do it as individuals. Do it as groups and let us know the results.”

For many years, such “experiments” were routinely shot down in court. “There exists a fundamental difference between depriving a prisoner of privileges he may enjoy and depriving him of the basic necessities of human existence,” one judge wrote, in response to a 1974 suit in Arkansas over a slew of deplorable prison conditions, including abusive solitary confinement. Three years later, a New Hampshire district court ruled that the state could no longer keep prisoners in isolation for more than 14 days. Protracted solitary confinement, the judge said, constituted “a psychological and a physical horror with the potential of devastating psychic, emotional and physical damage.”

But the judicial tide turned in the wake of William Rehnquist’s 1986 appointment as chief justice of an increasingly conservative Supreme Court. The very next year, in Turner v. Safley, a 5–4 court set a striking new standard by ruling that prison regulations impinging on “inmates' constitutional rights” were valid if “reasonably related to legitimate penological interests.” Over time, this standard gave prison officials extraordinary discretion to do more or less as they pleased. It was deemed acceptable to deny Muslim prisoners access to communal prayer on Fridays, to deny prisoners access to a law library or legal aid to help with their ongoing legal battles, and to restrict visitation rights at will. In a 2003 opinion, Clarence Thomas stated outright that he did not consider the denial of visitors a punishment. He even made the case that isolation, as conceived under the 19th-century Pennsylvania system, was not a punishment either; it was merely “a special setting for the ‘deviant’ [.].. where he would be placed in an environment targeted at rehabilitation, far removed from the corrupting influence of his family and community.”

This Orwellian redefinition of punishment — what Colin Dayan of Vanderbilt University has termed the Supreme Court’s “allowable suffering paradigm” — has given prison officials license to treat prisoners far beyond the limits of decency, even to consider them beyond rights at all. When a similar standard was applied to the “enemy combatants” at Guantanamo Bay, it was commonly held to be a new departure from the US’s constitutional norms, and provoked widespread indignation both domestically and overseas. In truth, though, it wasn’t a new idea, because the US prison system had already been denying basic rights to inmates, and finding justification to do so from the highest court in the land.

***

Prisons, as we learn from Dostoevsky and Foucault, can be metaphors for the nature of society at large. The metaphor now appears particularly chilling: we live in world of unprecedented surveillance and control, beyond the accustomed democratic checks and balances, where certain categories of people are now disowned and subject to shocking punishments beyond our view. When a federal judge examined conditions at Pelican Bay in 1995, the prisoners exercising alone in their yards left an impression “hauntingly similar to that of caged felines pacing in a zoo.” And he found the prison’s conditions hovering “on the edge of what is humanly tolerable.” Yet he felt constitutionally unable to close the prison or order any significant modification, because of the “legitimate penological interest” standard imposed by the Supreme Court.

This summer’s hunger strike in California has at last drawn broad public attention to the lengths to which the state is prepared to go in the name of crime and punishment, in much the same way that the scandals associated with Abu Ghraib and Guantanamo and CIA black sites have provoked debate about the limits of democratic accountability post-9/11. Prison reform in California, and the rest of the United States, is likely to be an arduous battle, made up of a patchwork of court rulings and protracted public debate about the cost and purpose of maintaining the biggest prison population in the Western world. But at least we can no longer say that we do not know what is happening under our noses. We are subjecting many thousands of prisoners to unbearable psychic torture day after day after day, at great expense and with no apparent benefit. And then we claim with a straight face that we are not in fact punishing them at all.

This article originally appeared in the LA Review of Books.