In London’s East End, where so many battles against real fascism were fought in the 20th century, “anti-racism” has become little more than a swindle. Far from being just or noble, it was a pretext to bribe journalists, pay off accomplices and frighten poor immigrants into supporting a crooked demagogue, who despised his “own” people so much he would not even grant them the right to participate in an honest election.

The formal reasons judge Richard Mawrey gave for disqualifying Lutfur Rahman from office last week are bad enough. The now ex-mayor of Tower Hamlets used fake “ghost” voters to win elections and public funds to buy votes. He offered grants to groups “that hadn’t even applied for them”. He took money that was meant to be going to the Alzheimer’s Society and poor wards that needed all the help they could get. He ran a “ruthless and dishonest” campaign to convince the electorate that John Biggs, his Labour rival for mayor, was a racist. When the election court asked Rahman if he believed for a moment that Biggs was an actual racist, he dodged the question. No matter. The truth of the charge didn’t worry him. His only concern was getting the lie out, and seeing it taken up by the local Bengali TV stations, five of which received public money from the mayor.

Not content with that, he rigged the vote by using “undue spiritual influence”, an accusation unheard of in a British court since the 19th century. Rahman persuaded clerics to go far beyond saying they thought he was the best candidate. Islam is under threat, they said in so many words. It was the duty of all Muslims to vote for Rahman. If Bangladeshi voters did not, they would be siding with their Islamophobic enemies, perhaps even defying God’s will.

We are used to thinking of racism as Nigel Farage or the Tory tabloids egging on their readers to see the Aids-afflicted foreigner as the enemy. Indeed, it often appears that this is the only way we can think about it. The mirror image is just as foul and its foulness reached a nadir in London. The worst of Rahman’s corruption was not the purloined money, but the way he corrupted leftwing values.

Anyone who criticised the mayor was a racist. When councillors said the mayor must answer questions, his supporters accused them of “racism”. When an opponent appeared at a meeting in a black cardigan – the poor woman was in mourning for her dead husband, incidentally – Rahman’s fixer roared that where once the East End had been terrorised by Blackshirts, it was now terrorised by Blackcardigans.

When Labour ran a candidate against him it was racist. When the BBC investigated him it was racist. And not just casually racist either. The judge noticed how Rahman always upped the ante by saying that all who tried to hold him to account were aiding the English Defence League. The EDL is, in truth, an ugly but small organisation that is close to collapse. For Rahman it was a gift. He could use it to paint his opponents as the willing accomplices of neo-fascists on the one hand, while corralling Bangladeshis frightened of racist attacks into line on the other.

“Truly, in Tower Hamlets,” the dry judge said , “if the EDL did not exist, it would be necessary to invent it.”

The neurotic fear of accusations of race and religious bias helped Rahman build a municipal dictatorship. The system of elected mayors is always open to abuse, because there are so few controls on them. Rahman pushed it to the limits. He controlled grants and officials could not prevent him handing public money to his supporters. He controlled the officials, too, and used supposedly impartial public servants to “carry out electoral activities on his behalf”.

Tower Hamlets First, his political party, was nothing more than a cult of the personality. If you wanted a safe seat on the council, you had to show a lapdog loyalty to Rahman. Speaking of dogs, the judge noticed that when there was not even the slightest justification for an accusation of racism, Rahman and his cronies would accuse their opponents of “dog-whistle politics” instead. By these means, anything and everything an opponent said could be turned into coded racism, even when the racism was only in the mind of the accuser.

Come on, admit it – it’s not just in the East End you see these tricks played. The postmodern universities and identity-obsessed scour speech for the smallest hint of bigotry, real or imagined. They seize on it – and with a whoop of triumph – cry that the mask has slipped to expose the true face of prejudice. Surely you have noticed, too, that in the paranoia that follows, careerists and charlatans flourish.

Do not forget either that Rahman at all times enjoyed the mulish support of Ken Livingstone and elements of what now passes for the British left. The BBC, the Daily Telegraph, Private Eye and Ted Jeory, a fantastic Tower Hamlets reporter, who exposed on his blog the corruption stories that local papers wouldn’t print, fought back. But with honourable exceptions, London’s leftwing press ignored the stink in its own backyard and dismissed the accusations against Rahman as evidence of a “deep substrate of” – you guessed it – “racism”.

You might think that at least the Labour party stood firm. But it left it to four Tower Hamlet residents to take on the huge financial risk of fighting Rahman. The judge wondered whether “like so many others who have come up against Mr Rahman, the party was not prepared to risk the accusations of racism and Islamophobia that would have been bound to follow any petition”.

One day, leftists and the Labour party will pay a price for their neglect and double standards. As it is, the price is being paid by others. Despite bordering on the opulent City, the East End of London is one of the poorest places in Britain. Many of its residents have no education; large numbers of Bangladeshis cannot speak English. They are wide open for hucksters to target. Too many stood back while they were shaken down, while money intended for them was diverted and their right to vote subverted.

In the onlookers’ indifference we can find, at last, an authentic white racism amid all the phoniness: the racism that believes the immigrant poor deserve no better.

Nick Cohen won the 2015 European Press Prize commentator award