Soon after​ Michael Brown was shot in Ferguson, Missouri, a book called The Short and Tragic Life of Robert Peace was published, describing one New Jersey man’s dual existence as a top student at Yale and an incorrigible drug dealer. Peace was an alarmingly precocious black boy whose mother toiled in hospital kitchens to raise the money to send him to parochial schools, where he thrived. His father, a magnetic hustler his mother refused to marry, was an active presence in his early life; he taught his son how to use his fists and decode the logic of the streets. When Peace was seven, his father was convicted of double homicide on circumstantial evidence, and sentenced to life in prison.

No one could claim that Peace had an easy path. Yet it’s also hard to deny that the institutions of US society unfailingly worked for him. Jeff Hobbs, who was Peace’s roommate at Yale, shows that at every stage of Peace’s life, his gifts were not just recognised but cultivated. He may have started selling marijuana to help his mother pay the rent, but his family didn’t have the crippling debts that frequently end any possibility of class mobility. He was the valedictorian at his prestigious high school, and a wealthy banker, moved by his speech, offered to pay all the expenses at whichever university he chose. He studied microbiology at Yale, but never stopped selling or using drugs. In 2011, at the age of 30, he was the victim of a gangland execution.

In the conversations about the deaths of Brown, Peace and numerous others who have commanded public attention in the US over the past year, there’s often a tension between the desire to attribute responsibility for actions to those who undertake them and the protective urge to downplay those same people’s responsibility for their actions. In Brown’s case, many people, including plenty of blacks, saw the predictable if gratuitous death of a young man who had committed a crime and then defied a cop; others saw this view as naive: it didn’t matter what the 18-year-old had or hadn’t done, because he wasn’t a moral agent in the first place.

In this second view, which is steadily gaining purchase in the US, Brown was a casualty of a centuries-old system of oppression that decided his fate before his parents’ parents had even met. This is the view held by Ta-Nehisi Coates, a 40-year-old journalist at the Atlantic, who makes the case most seductively in his recent memoir, Between the World and Me. Formally modelled on the first part of James Baldwin’s The Fire Next Time, the book is addressed to Coates’s teenage son, Samori, on the occasion of the non-indictment of Brown’s killer, a white police officer called Darren Wilson. ‘We are captured, brother, surrounded by the majoritarian bandits of America,’ he writes in a typically despairing passage:

The terrible truth is that we cannot will ourselves to an escape on our own … You have been cast into a race in which the wind is always at your face and the hounds are always at your heels … The plunder of black life was drilled into this country in its infancy and reinforced across its history, so that plunder has become an heirloom, an intelligence, a sentience, a default setting to which, likely to the end of our days, we must invariably return.

Between the World and Me traces Coates’s youth in a West Baltimore neighbourhood every bit as dominated by violence and drugs as the one where Peace grew up. The narrative focuses the experience of black subjectivity in the visceral, constructed fact of ‘the body’, a device derived from Foucault by way of feminist discourse. Coates returns again and again to his desire to ‘unshackle my body and achieve the velocity of escape’. His own family proved a haven of stability, but he writes affectingly of his parents’ harsh discipline, of watching his father prepare to beat him with a belt ‘in a kind of daze, awed at the distance between punishment and offence’. By his own admission not a particularly good student (‘the classroom was a jail of other people’s interests’), he did well enough to enrol at Howard University, a historically black institution in Washington DC, which he describes as ‘a machine, crafted to capture and concentrate the dark energy of all African peoples and inject it directly into the student body’. There, Coates first encountered a range of black people from different classes, regions and ethnicities, from the ‘scions of Nigerian aristocrats’ to ‘the high-yellow progeny of A.M.E. preachers’ and ‘California girls turned Muslim, born anew, in hijab and long skirt’. He fell in love with Samori’s mother, a young woman from Chicago, and befriended a handsome doctor’s son called Prince Jones, whose untimely death at the hands of the police functions as the moral fulcrum of the book. Coates then writes of his early years of struggle as a freelance journalist, but says little of his subsequent triumphs at the Atlantic, where since 2008 he has published a series of deeply ambitious and feverishly celebrated articles on the subject of race.

Hobbs, who is white, enjoyed a respectable success with his conflicted portrait of a single, flawed black life and the community that shaped it, but Coates’s bleak cri de coeur went straight to the top of bestseller lists and he achieved a status – it’s difficult to find an article on race in America today without a reference to his work – that’s rare for any writer. The book has just won the National Book Award and Coates recently received a MacArthur Foundation ‘genius’ grant. Coates has said that his book was originally conceived as a collection of essays about the Civil War, and that it was the death of Brown, and his relatively privileged son’s incredulous reaction to the failure to indict anyone for his killing, that caused him to shift focus, and allowed him to write with the withering heat of a righteous indignation ‘that burned in me then, animates me now, and will likely leave me on fire for the rest of my days’.

Between the World and Me is an unrelentingly severe, taut and timely text that’s been nearly universally praised. Before it was even published, the New Yorker’s editor, David Remnick, called it ‘extraordinary’; the New York Times critic A.O. Scott said it was ‘essential, like water or air’; in a widely circulated blurb, Toni Morrison likened Coates to Baldwin and declared the book ‘required reading’; even Jay-Z tweeted to his millions of followers: ‘Please. read.’

For some time now, Coates has been writing in what he and the avid readers of his blog (whom he calls, apparently with affection, ‘the horde’) have described as a ‘blue period’, rejecting any optimism about or faith in the possibility of national redemption: ‘I view white supremacy as one of the central organising forces in American life,’ he wrote on his blog in 2014, ‘whose vestiges and practices afflicted black people in the past, continue to afflict black people today, and will likely afflict black people until this country passes into the dust’. In the best of his recent journalism, such as his 2012 essay ‘Fear of a Black President’ or ‘The Case for Reparations’, which last year singlehandedly revived the long-defunct subject of affixing a financial value to the wrongs done to ‘those on whose labour and exclusion the country was built’, Coates has illustrated the intricate ways in which past racial inequalities continue to affect America today. But his less convincing and more doctrinaire efforts, such as the 17,000-word ‘The Black Family in the Age of Mass Incarceration’, which appeared in September, seem to be powered by a diffuse sense of outrage in ardent search of an affront. He presumes to speak on behalf of ‘the black family’ but erases the existence – extensively documented in Michael Javen Fortner’s recent study The Black Silent Majority – of the working and middle-class black families that for decades actively supported and sometimes participated in the implementation of many of the most notorious tough-on-crime measures that helped put the current US prison system in place.

Last year Coates claimed that Melissa Harris-Perry, a Wake Forest University professor and host of a Sunday cable TV programme, was ‘the foremost public intellectual in America’, and when challenged on this by a white journalist called Dylan Byers, responded as if there was no possible justification for Byers’s disagreement:

I came up in a time when white intellectuals were forever making breathless pronouncements about their world, about my world, and about the world itself. My life was delineated [on] lists like ‘Geniuses of Western Music’ written by people who evidently believed Louis Armstrong and Aretha Franklin did not exist. That tradition continues. Dylan Byers knows nothing of your work, and therefore your work must not exist. Here is the machinery of racism – the privilege of being oblivious to questions, of never having to grapple with the everywhere; the right of false naming; the right to claim that the lakes, trees and mountains of our world do not exist; the right to insult our intelligence with your ignorance. The machinery of racism requires no bigotry from Dylan Byers. It merely requires that Dylan Byers sit still.

The rhetorical leap that turns a cable TV host into black people’s ‘lakes, trees and mountains’ is breathtaking. It betrays either a cynical or a woefully skewed way of seeing the world.

Such missteps have barely registered, however, because Coates’s work exemplifies a broader shift in the rhetoric about race in America during Obama’s presidency. It may even have started in the latter half of the Bush administration, with the spectacular images of indigent black suffering that emerged from New Orleans in the aftermath of Hurricane Katrina. The brief honeymoon phase of ‘post-racial’ euphoria that followed the election of the first black president in 2008 quickly gave way to a sequence of events that showed that such a celebration was premature: in 2009 the Harvard professor Henry Louis Gates, Jr was arrested and handcuffed by a white police officer while attempting to open his own front door; in 2012 Trayvon Martin, an unarmed 17-year-old on his way home from a convenience store in Florida, was murdered by a neighbourhood watchman called George Zimmerman; in 2014 the killings of Brown and Eric Garner, an unarmed man from Staten Island who was choked to death by a white police officer, revitalised the Black Lives Matter movement. It was also during this period that the liberal papers that have championed Coates’s work began to drop the term ‘racism’ in favour of ‘white supremacy’, which is seen to refer not merely to individual acts or prejudices but to a presumed superstructure that keeps blacks and other ethnic groups permanently subordinate to whites, a shift in terminology that mirrors feminism’s move towards ‘patriarchy’ from ‘sexism’.

The more Coates, an earnest autodidact who dropped out of Howard to pursue his career in journalism, studied American history to research the book he intended to write on the Civil War, and the more he watched a procession of unarmed black men and women being attacked and murdered by police and vigilantes, the more he became convinced, as he told an interviewer in 2014, that ‘there’s nothing wrong with black people that the complete and total elimination of white supremacy would not fix.’ The essential premise of Between the World and Me is that blacks in America live entirely conditional lives. Yet when critics have addressed themselves to Between the World and Me, they’ve mostly danced around this and fixed instead on a single inconsequential, if somewhat bizarre passage in which Coates writes of watching the World Trade Center smoulder on 9/11. His tone is what’s notable as he recalls feeling no pity for the police officers or even firefighters who died trying to save lives in the burning buildings: ‘They were not human to me,’ he writes. ‘Black, white or whatever, they were the menaces of nature; they were the fire, the comet, the storm, which could – with no justification – shatter my body.’

The passage illustrates Coates’s fatalism nonetheless. He presents racism and the racial disparities that still persist in the US despite the undeniable progress that has been made towards equal rights as utterly intransigent and impersonal forces, like a natural disaster, for which no one can be usefully held to account. For blacks, this means that the phrase ‘black-on-black crime’ is nothing but ‘jargon, violence to language, which vanishes the men who engineered the covenants, who fixed the loans, who planned the projects, who built the streets and sold red ink by the barrel’. One of the recurring motifs in the book is a young Coates standing in a 7-Eleven parking lot when another black boy is about to be jumped by a group of older teenagers. He watched passively as

a light-skinned boy with a long head and small eyes reached into his ski jacket and pulled out a gun. I recall it in the slowest motion, as though in a dream. There the boy stood, with the gun brandished, which he slowly untucked, tucked, then untucked once more, and in his small eyes I saw a surging rage that could, in an instant, erase my body … He had affirmed my place in the order of things.

The mature Coates doesn’t have a word of reproach for boys like this one. He mostly feels sorry for them: they were ‘girding themselves against the ghosts of the bad old days when the Mississippi mob gathered round their grandfathers so that the branches of the black body might be torched, then cut away’.

It’s not​ just black kids in tough neighbourhoods who are hapless automatons. In Coates’s view, no one has agency. The young black shooter doesn’t have to think too hard about what he might do because ‘the galaxy was playing with loaded dice.’ What’s alarming, though no doubt comforting to his white readership, is that in this analysis whites aren’t individual actors either. When an irritable white woman leaving an Upper West Side cinema pushes the young, ‘dawdling’ Samori and impatiently screams, ‘Come on!’ Coates, who is a tall, imposingly built man, erupts:

There was the reaction of any parent when a stranger lays a hand on the body of his or her child. And there was my own insecurity in my ability to protect your black body … I was only aware that someone had invoked their right over the body of my son. I turned and spoke to this woman, and my words were hot with all of the moment and all of my history. She shrunk back, shocked. A white man standing nearby spoke up in her defence. I experienced this as his attempt to rescue the damsel from the beast. He had made no such attempt on behalf of my son. And he was now supported by other white people in the assembling crowd. The man came closer. He grew louder. I pushed him away. He said: ‘I could have you arrested!’ I did not care. I told him this, and the desire to do much more was hot in my throat.

Coates sees this woman not as a morally fallible person with her own neuroses, but as a force of nature, she is ‘the comet’ in his scheme. It doesn’t occur to him that she may not be an avatar of white supremacy but just a nasty person who would have been as likely to push a blonde child or a Chinese one. Coates doesn’t realise that his disproportionate reaction – ‘my words were hot with all of the moment and all of my history’ – is bound to be seen as objectionable to those ‘standing nearby’. And it doesn’t strike him that as long as black people have to be handled with infantilising care – for fear of dredging up barely submerged ancestral pain – we’ll never be equal or free.

This scene comes late in the book, and it’s the first, worst and only negative thing we actually see white people do to Coates or his family. The event that figures as the moral centre of Between the World and Me is the shooting of his Howard classmate Prince Jones by a police officer; the officer was black, as were most of the people who ran the community the officer served. The other white people who appear in the book are interviewing Coates on television, helping him get a foothold in his career, or taking him out to eat charcuterie. (In the last instance, he doesn’t actually say that the Parisian who picked up the tab was white, but he does confess that he was so anxious about being set up by such gratuitous kindness that he couldn’t enjoy his meal.) Coates admits that he didn’t really know any white people when he was a child: he only saw them on television. That is an indictment of the segregation that still exists in many American neighbourhoods, but it’s two decades since Coates left Baltimore and if his current writing is any indication, he still doesn’t know white people very well. He tends to attribute to them greater power and satisfaction with their lives than most of them have. He writes that whites ‘think that they are beyond the design flaws of humanity’, although the profits of the pharmaceutical or self-help industries might suggest otherwise. In Paris (where I live) he sees cartoon figures: ‘the men in salmon-coloured pants and white linen and bright sweaters tied around their necks, the men who disappeared around corners and circled back in luxury cars, with the top down, loving their lives. All of them smoking. All of them knowing that either grisly death or an orgy awaited them just around the corner.’

This nonsense, a fantasy that flattens psychological and material difference within and between groups, serves only to bolster Coates’s sense of injury. The late Albert Murray, whose under-appreciated masterpiece, The Omni-Americans, is one of the best commentaries on modern African-American life, argued in the 1970s that ‘no matter how noble your mission, when you oversimplify the reasons why a poor or an oppressed man lies, cheats, steals, betrays, hates, murders or becomes an alcoholic or addict, you imply that well-to-do, rich and powerful people don’t do these things. But they do.’ Of course they do.

One of the rhetorical moves Coates uses to great effect – positioning himself permanently as a member of the ghetto where he grew up – is to conflate blackness (and conversely whiteness) as a physical and cultural designation with an economic and political position. As I read Between the World and Me, I was swept away by the incantatory momentum of the prose, but I was troubled by what was left out in the rush to take umbrage. In Coates’s telling, an essential part of the story of black life today – the only black life I have ever known – is missing. The capacity of humans to amount to more than the sum of a set of circumstances is ignored. The capacity to find gratification in making a choice – even if it’s the wrong one – is glossed over. Yet even Coates admits these pleasures:

The streets transform every ordinary day into a series of trick questions, and every incorrect answer risks a beatdown, a shooting or a pregnancy. No one survives unscathed. And yet the heat that springs from the constant danger, from a lifestyle of near-death experience, is thrilling. This is what the rappers mean when they pronounce themselves ‘addicted to the streets’ or in love with ‘the game’.

I never knew​ Robert Peace, but he was a year older than me and grew up in East Orange, a bleak New Jersey town near Newark, where numerous high school classmates of mine lived. Many of the black boys and girls I was close to faced discrimination and some made disastrous choices that the interpersonal and structural racism we encountered can’t alone explain. Many of us were enthralled by what the Harvard sociologist Orlando Patterson has called ‘the Dionysian trap’. For the group of young black men that Patterson and his colleagues were studying, ‘it was almost like a drug,’ he writes, ‘hanging out on the street after school, shopping and dressing sharply, sexual conquests, party drugs, hip-hop music and culture, the fact that almost all the superstar athletes and a great many of the nation’s best entertainers were black.’ Here, I see my younger self, a black self that Coates is loath to consider because to include it would be to recognise that the ‘thrill’ is not the sole province of those on the ghetto front line. To admit the existence of such young men would be to recognise that some of this, perhaps a great deal, is a matter of choice, and available throughout the culture.

At what point might an oppressed group contribute – perhaps decisively – to its own plight? This is a terrible question that any analysis of black life must eventually confront. Academics and others on the left have for some time been ‘allergic to cultural explanations’ for the behaviour of young blacks, Patterson argues, holding that such analysis is equivalent to blaming the victim for his or her own condition. He recognises that the unwillingness to accept such explanations is exacerbated by the way the political right often disingenuously appropriates the language of values to excuse or deny the persistence of racial injustice. But that can’t invalidate the argument, and understanding the role black youth culture plays doesn’t cancel out the significance of the historically brutal social conditions that writers like Coates so eloquently evoke. ‘It is impossible,’ as Patterson writes, ‘to understand the predatory sexuality and irresponsible fathering behaviour of young black men without going back deep into their collective past.’

One of the great ironies Patterson draws attention to is that, throughout the ‘forty-year period of self-imposed censorship’ within the discipline of sociology on the subject of black culture in the wake of Daniel Patrick Moynihan’s report The Negro Family: The Case for National Action and Oscar Lewis’s ‘culture of poverty’ theories,

the vast majority of blacks, and especially black youth and those working on the front lines of poverty mitigation, have been convinced that culture does matter – a lot. Black youth in particular have insisted that their habits, attitudes, beliefs and values are what mainly explain their plight, even after fully taking account of racism and their disadvantaged neighbourhood conditions. Yet sociologists insisted on patronisingly treating blacks in general, and especially black youth, as what Harold Garfinkel called ‘cultural dopes’.

It’s not just sociologists who would paint us all – white and black and everything in between – as dopes who ‘simply reproduce society without being aware of it’. Many black people have a far more nuanced understanding of the Michael Brown shooting, for instance, than narratives like Coates’s allow for. In Between the World and Me that vexed encounter between Brown and Wilson appears again and again as a self-evident instance of the permanence of racial injustice, and nothing else. ‘That was the week you learned that the killers of Michael Brown would go free,’ he writes to his son. ‘It was not my expectation that anyone would ever be punished. But you were young and still believed … Here is what I would like for you to know: in America, it is traditional to destroy the black body – it is heritage.’

The acceptance of this pessimistic assessment means that forty million people must be seen as permanent victims. That would have alarmed black intellectuals of previous generations – including James Baldwin, whose authority Coates frequently invokes. ‘It seemed to me that if I took the role of a victim then I was simply reassuring the defenders of the status quo,’ Baldwin told the Paris Review shortly before he died. ‘As long as I was a victim they could pity me and add a few more pennies to my home-relief cheque. Nothing would change in that way … It was beneath me to blame anybody for what happened to me.’ Baldwin’s own solution to the racial sickness that has never stopped plaguing the US – a laborious, courageous appeal to love – hasn’t attracted many followers, but it’s easy to see why Coates’s remedy is so alluring, both for disillusioned blacks who’ve found a fiery advocate and even more for well-meaning whites: if the galaxy really is ‘playing with loaded dice’, they don’t have to do anything other than read Coates’s blog and nod.

‘Need my skin blind me to all other values?’ an exasperated Ralph Ellison wrote in 1963. It’s a question Coates and many others wouldn’t think to ask today. The crisis of the black intellectual now, if there is one, isn’t that he lacks the means or the platform to represent his people but that it is too easy to cleave to a sense of resentment and indignation – even now that he has found himself, after all these years and all this struggle, in a position of strength.