YOUNG black men out in the streets at night, lines of police officers dressed for a riot, cars ablaze, stores looted, the morning-after platitudes about coming together: the recent scenes in Baltimore (pictured) recall similar ones in Ferguson, Missouri in 2014, Cincinnati in 2001, Los Angeles in 1992 or half a dozen cities in 1968.

The Baltimore riots were sparked by the death of Freddie Gray, a black man, in police custody. But the underlying cause was more complex. Wali Uqdah, a retired prison officer in the city, says: “It’s a building up of hostility. It’s not about just one incident; it’s like if I leave a pot on my stove, and I go outside, it’s just going to get hot and hot until it boils over. There’s no jobs, no income, no good schools…”

Many Americans feel a confused sense of guilt when the problems of poor black neighbourhoods come to their attention, unsure whether the persistence of crime and poverty in such places is, in some convoluted way, their fault or the fault of the people who live there, and unsure what can be done about it. On the one hand, they see that the police are sometimes racist. On the other, they note that tensions between blacks and cops persist even in cities like Baltimore that have a black mayor, a black police chief and a mostly non-white police force. Shortly after the riots, six officers were indicted for abusing Mr Gray, on charges including second-degree murder. Three of the cops were black.

This year marks the 50th anniversary of a bold and controversial attempt to explain what has gone wrong in America’s inner cities: Daniel Patrick Moynihan’s “The Negro Family: The Case for National Action”. Moynihan, then a bureaucrat in the Department of Labour, made two main points. First, he argued that the lingering effects of two centuries of slavery had undermined the black family—at the time, 25% of black babies were born to unmarried mothers (see chart 1). Second, he argued that family instability was at the root of many other problems, from crime to poverty.

Fifty years later, black America still fares badly on many of the predictors of success and signals of distress that concerned Moynihan. If it were a separate country, it would have a worse life expectancy than Mexico, a worse homicide rate than Ivory Coast and a higher proportion of its citizens behind bars than anywhere on earth (see interactive). This is despite the fact that, overall, America is home to the richest, most successful population of black African descent that the world has ever seen.

America is also far less racist than it was in Moynihan’s day, when interracial marriage was still illegal in 19 states. Now it has a black president—who won more of the white vote in 2008 than John Kerry, a white Democrat, won in 2004. The census form today allows people to identify themselves as white and black, too. In 2010 over 2m did so, breaking through the thickest wall in American history with a few strokes of a pen.

Yet an updated Moynihan report would also have to acknowledge the prescience of that tall white Irishman. The proportion of African-American babies born outside marriage has nearly tripled since 1965, to 71%. Though the crime rate has fallen across the country in the past two decades, casting some doubt on Moynihan’s link between single-parenting and disorder, black Americans are still eight times more likely to be murdered than whites and seven times more likely to commit murder, according to the FBI. An incredible one-third of black men in their 30s have been in prison. Blacks are also less likely to graduate from college than whites, and less socially mobile (see chart 3).

Historians have gone back and forth on the link Moynihan made between slavery and the fragility of the black family, but many now agree with him. “The obvious explanation turns out to be the right one,” says Orlando Patterson of Harvard, who points out that every black population in the Americas has low rates of two-parent families. “If you have many generations in which you are not permitted to have a relationship, in which you have no custodial rights to your children or spouse and your family members can be sold away, that has an effect,” says Mr Patterson. Yet events in the 19th century cannot explain why black families have grown so much more fissile since the 1960s, nor why white families now have non-marital birth rates as high as the black ones that so alarmed Moynihan in 1965. Slavery may have started the dissolution of the black family, but something else must have accelerated it.

Because the nationwide decline in marriage began at around the same time as Lyndon Johnson’s Great Society programmes, some argue that welfare is the culprit. Social conservatives complain that for many families, a welfare cheque has replaced the male breadwinner, making him superfluous. Worse, single mothers who tie the knot are often penalised, since the addition of a father’s income to the household total causes all manner of means-tested benefits to be withdrawn. However, studies looking at how marriage varies between states with different rules on eligibility for Medicaid, one of the largest of the many means-tested programmes, have failed to find a link. “I would never say welfare plays no role,” says Isabel Sawhill of the Brookings Institution, a think-tank, “but it’s not the main thing.”

Analysts on the left argue that the collapse of the black family is largely a consequence of slumping wages for unskilled black men, which makes them less attractive as mates. But this, too, can only be a partial explanation. As incomes rise, Americans of all stripes are more likely to get married. But even accounting for this, black families look different from white ones. Black women with advanced degrees are about as likely to be single mothers as white women with a high-school diploma. The gap between black and white families only disappears once the household incomes of black families rise above $200,000 a year—a tiny minority.

Another explanation for the low marriage rate among African-Americans is that many black men are “missing”, either because they have died early or because they are in prison. For every 100 non-incarcerated African-American women aged 25-54, there are only 83 black men, according to the New York Times. For whites, the ratio is 100 to 99: ie, there is hardly any gap at all.

Kathy Edin of Johns Hopkins University interviewed hundreds of inner-city single mothers. Many told her that having a child was a way to give purpose to life and (they hoped) hold on to a boyfriend. For the men she spoke to, fatherhood was a source of pride. For the children of such unions, that may be scant comfort.

Growing up in a single-parent family makes life harder for most children. The Fragile Families study run from Princeton and Columbia universities, which examines how children born to single mothers fare, has found that 30% have had two or more father figures in their home by their fifth birthday. Some 40% of households headed by single mothers are poor; for two-parent families the proportion is 9%. Those numbers, which put black children in a precarious place, are made worse by the places where black families live. And there is nothing random about that.

Most of the black Americans who are now struggling are the left-behinds of two great internal migrations. The first, from south to north in the early 20th century, left behind people in the Mississippi Delta, which is now the poorest bit of the country. More recently a second migration has been going on, from northern cities to southern ones. For the past two decades Georgia has attracted more black migrants than any other state. This smaller exodus has left people behind in highly segregated bits of northern cities—such as West Baltimore, which is 96% black. Despite living in some of the most benighted places in the country, African-Americans are the least likely to move of any ethnic group.

These left-behind neighbourhoods, which can be found in Milwaukee, Detroit, Philadelphia, Washington, DC, New York, Chicago and dozens of smaller cities, are the places where black migrants were funnelled in the mid-20th century under racist housing policies. Baltimore was the first city to formalise residential segregation by race, but others soon followed. In 1942, with black GIs preparing to go to war, 84% of white Americans told pollsters that “there should be separate sections in towns for Negroes.” De jure racial segregation is now a thing of the past, and de facto segregation has declined since 1970. But it has not done so evenly: on a scale where 0 means blacks are evenly distributed and 100 means they live completely separately, Milwaukee scores 82, New York 78 and Chicago 76. Anything above 60 is high.

Dante Washington, a Baltimore native in his 30s, grew up in one of these places, in a house passed down by his grandmother to his mother in a neighbourhood where gunfire began when the sun set. He recalls being in his bedroom one night, hearing shots in the street outside and not bothering to look. The next morning he discovered that the brother of his best friend had been killed. Another friend was murdered on a basketball court for no clear reason. Mr Washington graduated from college and now works as a publisher in a suburb of the city that has a Lexus dealership and where the dogs yap rather than snarl.

Stories such as his are far too rare. Karl Alexander and his colleagues at Johns Hopkins studied 790 six-year olds who entered Baltimore public schools in 1982, following them for the next 22 years. Of the kids who started at the bottom, in low-income families where the parents had a combined total of ten years’ schooling, only 4% graduated from college.

The house Mr Washington’s grandmother left his mother now has seven boarded-up properties for neighbours. In West Baltimore whole blocks have gone: a tree that has grown inside one house is visible through the glassless upper-storey windows. Plenty of houses are worth nothing or, if they have tax bills owing, even less. The whole area consists of 62,000 people (within a prosperous metropolitan area of 2.7m). The murder rate is 97 per 100,000—20 times the American average (see chart 4).

The gap between what black and white families earn is large enough. The wealth gap is much larger: the median white family in 2013 had net assets of $142,000; the median black family had a paltry $11,000 (see chart 5). Wealth gaps are nearly always bigger than income gaps, since people who earn more find it easier to save. For black households, this is often exacerbated by absent fathers. A one-parent family with the same income as a two-parent family is probably spending a lot more of its spare cash on child care. Saving is extremely hard under such circumstances, which is one reason why black families find it harder to buy a house than white families with the same income, and why black university students rack up larger debts. Some 60% of black students have yet to complete a four-year degree after six years, compared with 37% of whites. Some cannot afford the books or the train fare or need to look after a sibling or work to support their families. “You start off wanting to be a policeman or a doctor,” says Mr Washington. “Eventually you just want to eat tomorrow.”

What if black Americans in the worst neighbourhoods were given a chance to move out? In 1966 the Chicago city government was taken to court for building all its public housing in areas that were wholly black. By way of compensation, it provided vouchers for 7,500 families to move to nicer (and whiter) parts of the city and its suburbs. Studies found that 15-20 years later the families were still in their new neighbourhoods and their children were attending better schools and doing better than those who stayed behind.

After the Los Angeles riots in 1992 the federal government tried to copy this scheme in other cities. Until recently, the results were considered disappointing. Those who moved out of public housing in crime-ridden places showed lower rates of diabetes than those who remained, and mothers who moved showed an increase in happiness similar to the effects of Prozac, an antidepressant. However, children did no better after moving and their mothers did not get better jobs.

In a paper published this month, however, Raj Chetty and colleagues at Harvard re-examined the numbers and found that the children who moved earned considerably more in their 20s. This was true only of children who moved before their 13th birthday: older children saw no benefit later on. More recently, Baltimore lost a public-housing case similar to the one in Chicago and, in 2005, provided vouchers for 7,000 families to move to other parts of Maryland. They are still there, and three-quarters of their children are attending much better schools, says Stefanie Deluca of Johns Hopkins. Yet since giving a golden ticket to every poor black family would cost about $30 billion a year, those stuck in highly segregated places will have to save in order to get out.

Government probably cannot do much to put broken families back together. But cities could change the way their police forces work, as Los Angeles and New York have done. Prosecutors and judges could explore alternatives to mass incarceration. School systems could give parents more choice, so their children can escape from dysfunctional schools. All these things and more could make marginal improvements that, when added together, would amount to something greater.

“That the Negro American has survived at all is extraordinary,” wrote Moynihan. “A lesser people might simply have died out, as indeed others have.” On the 50th anniversary of his report it is worth remembering that freedom, in the sense of being allowed to live where you choose, is still a recent acquisition for African-Americans. In another 50 years the lines that divide black and white America, which have begun to fade, will be fainter still.

Timeline picture credits: Corbis; Getty Images; AP