As riots in Ferguson and Baltimore heated up this past winter and spring, so did denunciations of a criminal-justice system that has placed a disproportionate number of black men behind bars. One widely aired theory holds that not only are racial disparities and mass incarceration patently unjust on their own terms, but they also lead to chaos in poor urban families. Black men’s absence “disrupts family formation, leading both to lower marriage rates and higher rates of childbirth outside marriage,” pronounced a widely discussed New York Times article, “1.5 Million Missing Black Men.” Hillary Clinton auditioned the theory in the first policy speech of her presidential campaign. “When we talk about one and a half million missing African-American men,” she said at Columbia University in April, “we’re talking about missing husbands, missing fathers, missing brothers.”

The missing-men theory of family breakdown has the virtue of being easy to grasp: men who are locked up are obviously not going to be either desirable husbands or engaged fathers. It also bypasses thorny and deadlocked debates about economics and culture. Still, it has a big problem: it’s at odds with the facts.

Consider the first graph (below), which traces the percentage of births to black unmarried women between 1960 and 2013. As the sixties began, 20 percent of all black births were to single mothers. By 1965, black “illegitimacy,” in the parlance of the time, had reached 24 percent and become the subject of Daniel Patrick Moynihan’s prophetic but ill-fated report “The Negro Family: The Case for National Action.” That number, startling at the time, turned out to be only the ground floor of a steep 30-year climb. By 1980, more than half of black children were born to unmarried mothers. The number peaked at 72.5 percent in 2010.

Graphs by Alberto Mena

Now look at the second graph, tracing the number of black men admitted to state and federal prisons over this same period. In the 1960s and early 1970s, as nonmarital births raced upward, mass incarceration was not yet a glimmer in policymakers’ eyes. The black prison population hovered at a relatively low level, showing no serious trend up or down. In 1975 came the first signs of a clear increase, but it wasn’t until 1980 that the numbers of black men admitted to state and federal prisons really mushroomed. (Though the graph doesn’t show it, the same trends were true for white men, though their prison-admission rate per 1,000 is far lower.) Throughout the 1990s and most of the first decade of the 2000s, the prison population grew to historical highs. In 2008, the black (and white) prison population peaked; since then, it has declined, though only slightly.

The two graphs together show that the black family was in deep disarray well before America’s experiment in mass incarceration. If anything, the timing of the two problems points to the opposite causation from the one assumed by “missing-men” theorists: as the family unraveled, crime increased—the homicide rate doubled between the early 1960s and late 1970s, with more than half of the offenders being black—leading to calls for tougher sentencing and more bad guys behind bars. In other words, family breakdown led to more crime and more crowded prisons. That theory also jibes with abundant research showing that boys who grow up without a father in the home are at greater risk of criminal behavior.

We shouldn’t take this alternative theory too far. Crime and prison rates are unlikely to have a single cause: demographics, policing and sentencing policies, environmental toxins, and who knows what else may all play some role. Perhaps the most controversial of those policies was what came to be known as the War on Drugs, first declared by President Richard Nixon in 1971. There’s little question that the government’s hard line on drugs eventually put large numbers of black (and, again, white) men behind bars.

However, if the War on Drugs played any role in shaping the contemporary black family, it’s almost impossible to decipher from the data. Despite Nixon’s early 1970s call to arms, the War on Drugs didn’t lead to many casualties until well into the next decade. As of 1979, only 5.7 percent of U.S. prisoners were incarcerated for drug offenses. Yet as we’ve just seen, by that time, nearly half of black births were already to single mothers. The number of men imprisoned for drug crimes rose only modestly until 1990, four years after Congress passed the Anti-Drug Abuse Act, legislating harsher sentences for crack cocaine, a move often cited as a cause of the disproportionately black prison population. Far from leading to more fatherless children, the growing number of black men imprisoned for drugs coincided with a flattening out of the percentage of black single mothers, after a 30-plus-year upward climb.

In fact, whatever its evils, the War on Drugs doesn’t take us far in explaining prison racial disparities. That’s not the impression you’ll get from the punditry class. “More than half of federal prisoners are incarcerated for drug crimes in 2010,” goes a typical formulation, from the Huffington Post. It’s true, as far as it goes—but “federal prisoners” make up only about 14 percent of all incarcerated men. In the far larger state system, the majority of black men are doing time for violent crimes. Between the federal and state system, almost two and a half times the number of black men are serving sentences for murder, assault, and the like than they are for using and selling drugs. Today, violent criminals continue to make up by far the largest cohort of the freshman class of prisoners—black, white, and Hispanic.

The preponderance of violent prisoners muddies another plank of the missing-men theory: that mass incarceration of black adults has harmed black children. Researchers have made a compelling case that when fathers go to prison, their absence takes a toll on their kids. Boys, especially, have more behavioral problems, including aggressive acting out and lower educational achievement. But the lessons of those findings are far from clear. You can construct a reasonable argument that the children of men sentenced for drug offenses—and the communities they live in—would be better off if fewer fathers were behind bars. When it comes to men prone to violence, though, that supposition is ambiguous, at best.

The difficult truth avoided by most missing-men adherents is that men doing prison time are part of a larger population that doesn’t provide much in the way of paternal care, even if they never were locked up. According to research from the Fragile Families project, the vast majority of poor black fathers are unmarried, though it’s also the case that those in jail are the least likely to be married. That same population is less likely to be living with their children and their children’s mother than unmarried white or Hispanic fathers, a fact that lessens their sons’ and daughters’ chances in life. It’s true that black nonresidential fathers have been found to be more involved with their kids in the early years than comparable white and Hispanic dads. But that only lasts until the kids are nine or so; after that, poor black fathers are actually more likely to have exited their children’s lives than other men—another blow to children’s well-being. Part of the reason for this is “multi-partner fertility”—social-science-speak for having children by more than one partner. Multi-partner fertility, with the complex families it creates, is extremely common in poor black neighborhoods. (Again, it’s twice as prevalent among incarcerated men.) Serial parental relationships, breakups, disappearances, a line of step-siblings and parents: they’re all part of what sociologists Laura Tach and Kathryn Edin call the “family-go-round.” These traits are linked to the same behavioral problems experienced by boys whose fathers have been jailed.

None of this means that incarceration policies aren’ t ready for an overhaul. The country needs a vigorous examination of mandatory-sentencing laws, the War on Drugs, and racial arrest and sentencing disparities. But that debate should not be used to evade the realities of family life in neighborhoods like Ferguson and Baltimore’s Sandtown. Evasion has been the preferred modus vivendi over the past 50 years, ever since Moynihan’s warning of rising fatherlessness in the ghetto drew sharp condemnation. Look where it’s gotten us.

Photo by Joe Raedle/Getty Images