‘I feel the pressure, under more scrutiny/ And what do I do? Act more stupidly,” rapped Kanye West on Can’t Tell Me Nothing. And act more stupidly he then did. He began by embracing Donald Trump as “a brother” and ended by suggesting slavery had been a “choice”.

The backlash was swift. From 50 Cent to Spike Lee to Roxane Gay, celebrities, scholars and seemingly half of Twitter pushed back, pointing out the imbecilic character of West’s comments on slavery. Not only had savage force been used to capture, transport and maintain transatlantic slaves, but, despite the brutality, slaves had constantly rebelled against their condition, heroically and at great cost.

But if West’s claims were idiotic, much of the response was equally so. The problem for many critics was not just what West said, but also that he was a black man saying it. His was an act of betrayal of the black community, indeed of his very blackness.

This argument was made most elegantly, and brutally, by the essayist Ta-Nehisi Coates. In an acerbic tear-down titled I’m not black, I’m Kanye, Coates recalled his mother’s response to Michael Jackson. He “was dying to be white”, she observed; he was “erasing himself, so that we would forget that he had once been Africa beautiful and Africa brown”.

Thirty years on, Coates has a similar response to West. Black music, he argues, is inextricably linked with black history and community. And West, like Jackson, is attempting to escape that history and community, to be not-black. West champions “a white freedom”, a “freedom to be proud and ignorant”. And, writes Coates, all blacks “suffer for this, because we are connected”.

Many on the left have long seen rightwing black or gay people or women as traitors to the cause. There is something disturbing in this claim that there is a right way of thinking for oppressed peoples and that those who dissent are committing betrayal.

It is a way of thinking about race, community and heresy that has deep, reactionary roots. “Traitors” is how Islamists describe liberal Muslims. It is how the apartheid government in South Africa explained white anti-apartheid activists. And it is the label that the far right has long hung upon white anti-racists. Thomas Mair, who murdered Labour MP Jo Cox in 2016, saw her as a “collaborator”. “My name is death to traitors, freedom for Britain,” he declared at his trial.

It may be comforting to imagine that if black people are being reactionary, then they are not really black, or at least they are attempting to escape being black, by espousing “white” ideas of freedom. But is it really less reactionary to imagine that ideas come colour-coded than it is to claim that slavery was a choice? Or any more progressive to insist that West is not black because he backs Donald Trump than it is to see Trump as a “brother”?

While some denounced West as a traitor, others insisted that whites should butt out of the debate. “If you think that you get to criticise black people for selling out to the system of anti-blackness that you as a non-black person benefit from and help maintain,” wrote Ijeoma Oluo, a Seattle-based writer, “you need to check your privilege and be quiet for a while.”

Whites, she added, should “stay in your own lane”.

This, too, is an old racist trope. “As a person of colour,” critic David Dennis wrote in a 2013 essay on West, “I’ve been told repeatedly to ‘stay in my lane’. From something as simple as being followed around my neighbourhood by police to my profession, where I’ve been told to stick to writing about ‘black stuff’ and leave the ‘real news’ to white writers.”

Where racists patrol the streets and the workplace to ensure black people know their place, a new class of “anti-racists” seek to police public debates to ensure that only the right people speak and only the right things get said.

Segregation of public debate in the name of “anti-racism” – and that is what the demand to “stay in your lane” amounts to – is no more progressive than the racist segregation of social space.

Any struggle against injustice requires us to get out of our lanes, to insist on the right, whoever one may be, to speak as we see fit, against every wrong.

As Kanye West and many of his critics have shown over the past weeks, there is more than one way of being reactionary.

• Kenan Malik is an Observer columnist