I'm in two minds about returning to "Starkeygate". The Tudor historian's comments on that episode of Newsnight were, from where I was sitting, so blatantly racially inflammatory (close friends instantly recognised my genuinely stunned expression when his diatribe began) that I was astonished that anyone would even attempt to defend him. Some did - largely by minimising his racism, unlike the white supremacists who have since filled my inbox with hate-filled bile ("you're a traitor to the white race", etc, etc).

David Starkey also enjoys the attention: I was warned beforehand that he loves to provoke controversy for the sake of it (this is a man who once called Scotland "a feeble little country"). His love of publicity is clear in his first article (in the Daily Telegraph) since Dreda Say Mitchell and I were ambushed with his ignorant bigotry in a BBC studio: he clearly relishes the fact that Ed Miliband joined the chorus of disgust at what he calls his "now-infamous opinions".

But as a historian, Starkey must surely object to attempts to revise the past, and that's why his self-justification in the Telegraph piece needs a response. Starkey is outraged that Miliband slammed him for making "racist comments", and then gives a few examples of what he said - without mentioning the key offending lines. But even his selective quotes distort what he said: "This sort of black male [gang] culture militates against education." Hold on a minute - can he really get away with inserting "gang" into that sentence - a word he did not use at the time in the studio and the inclusion of which completely transforms the meaning of what was said?

Starkey may be a bigot, but he is not stupid. He must surely understand why Miliband and others accused him of racism. On Newsnight, he argued that "the whites have become black". Funnily enough, he makes no mention of this in his article. Given we were discussing why people had become rioters and looters, this was a straightforward equation of being black and violent disorder. By becoming involved in the August riots, the white participants had somehow become black.

But in any case, as a historian, Starkey is aware that white people do not need to "become black" to become gangsters - the Kray brothers remaining the country's most famous examples. As for riots, they've taken place long before they were any significant numbers of non-white faces appeared on British soil - and that includes the Tudor period that Starkey specialises in. If we're just going to talk about riots in the post-war period, Starkey is surely aware of the 1958 Notting Hill riots, when groups made up of mostly young white men attacked black residents.

He had further suggested that, were you to listen to David Lammy - "an archetypal successful black man" - "you would think he was white". Again, this led to an obvious interpretation: to sound respectable was to sound white. In his article, Starkey attempts a convoluted defence: that the likes of Lammy and Diane Abbott "have merged effortlessly into what continues to be a largely white elite" and, in doing so, had lost "much of their credibility with blacks on the streets and in the ghettos". Of course, this raises other questions as to why Starkey thinks he's any authority on the attitude of black Britons towards prominent black politicians - but the bottom line is that none of this was mentioned in the studio.

And, of course, he began by suggesting a partial vindication of Enoch Powell, a politician who had argued that mass immigration would bring violent chaos to Britain's streets (a prophecy discredited by history). Powell was "absolutely wrong" about "inter-communal" violence, Starkey conceded. But the implication was that immigration had indeed brought disorder to Britain's streets - but by the unforeseen means of black people colonising white people with their culture. It was a means of scapegoating black people for riots that had involved people of all races. Starkey's friends apparently unanimously believe quoting Powell was an error. That's an understatement: putting the "Rivers of blood" speech on the political agenda at a time when people were angry and scared in the post-riot aftermath was outright dangerous.

Starkey lays the blame on "gangsta culture" in his piece. If he had done that in the studio, he would have been wrong, but it would have been an argument at least worth debating. But he was talking about black people and black culture more broadly.

His defenders have similarly misconstrued what he said: Toby Young argued that he "wasn't talking about black culture in general", but only a "sub-culture associated with a small minority of people of African-Caribbean heritage." Again, not what he said, and even Young was forced to admit "he could have made this clearer." My one-time sparring partner James Delingpole seems to imply I helped set the whole thing up: "it was a trap", he argued. I had apparently decided to add black people to my list of oppressed groups to take "perpetual umbrage and righteous rage on behalf of". The reality was both Dreda and I were taken unawares by a bigoted outburst, and had no choice but to respond. Perhaps more bizarrely was Howard Jacobson's argument that I had taken part in a "mugging"; I was a "baby-faced assassin", apparently - not a guest subjected to a series of outrageous comments who could barely get a word in edgeways. So, I should probably clarify that I did not compel Starkey to make racist generalisations.

Perhaps the only remotely thought-provoking element of Starkey's Telegraph piece is the suggestion, with white working-class culture facing a "systematic attack over several decades", the vacuum has been filled "with the values of 'gangsta' culture". But, to Starkey, to even listen to hip hop was to be part of "gangsta culture" ("do you glorify rap?", was a Brass Eye-style question he put to me). We know that there is a link between deprivation and gangs; it is this, not owning hip-hop CDs, that drives gangsterism.

We still need a debate about what caused these riots: and about the growing numbers of young people who feel they have no future to put at risk (not least in a country where over one in five 18-24 year olds are out of work and education). But racist comments by the likes of Starkey have no role in that debate. And - let's be clear - however Starkey and his allies twist what he said, his comments were racist. So let's not let the historian rewrite history.

Owen Jones is author of "Chavs: The Demonization of the Working Class"