Saturday night in London, Seven Sisters Road. A packed 253 bus, heading west from Finsbury Park, full of families, shoppers, young and old. A man boards, singing loudly as he walks down the aisle of the lower deck. His baritone bored its way upstairs, and while grating, the lyrics were too tame to warrant more than a shrug of annoyance. Until the tone changed, and the baiting began.

"You, you're a fucking bomber, you're here to blow us all up – and you, you're not from England, get off the fucking bus." He was spitting out venom at an ever-increasing rate, and those in his line of fire either moved out of harm's way near the driver, or reluctantly alighted to wait for the next 253 to arrive.

He was well into his stride by now, rounding on the remaining cowed passengers with increasing rage, until I jogged down the stairs and over to where he sat. I told him in no uncertain terms to keep his mouth shut, that we'd all heard more than enough and that his silence was long overdue. Inevitably, he turned on me, cursing my intervention before swivelling round to continue his abuse of a black mother and her children huddled in the back corner.

I told him again to shut up, he spat "make me," and jumped up, fists raised. Given his age, size, and my positioning, it was simple to restrain him, even as he offered to knife me and tried to reach for his left pocket. Emboldened by my lead, other passengers shouted to the driver to stop the bus and call the police to have the man removed, and I was tasked with restraining him in the interim.

The incident ended relatively calmly, but just watch the faces of all those aboard and consider the crushing effect he'd already managed to have on so many in such a short space of time. His outburst is just the tip of an ever-growing iceberg: YouTube lists scores of such London-based incidents, including the infamous Croydon tram race rant, and the bizarre tirade on the bus to New Addington. While new technology clearly increases the amount of capture and coverage of such vile attacks, the freedom with which the perpetrators can carry out their crimes only increases the chances of others being encouraged to mimic their performances.

For a city so defiantly touted as a model of multicultural co-existence, the lack of protection from abuse on London's public transport system is appalling. The abuse is not restricted to simply racist attacks, rather it spreads right across the antisocial behavioural spectrum, with passengers and Transport for London staff alike seemingly powerless to act in its wake. Fear of violent attack, or even of arrest for intervening, paralyses people and prevents them from responding – in my case, neither were a serious concern, given the man's age, size, and my military training in restraint without crossing a legal line.

But in more serious cases, involving younger or multiple aggressors, there is little chance of them being brought to task, or of other passengers being offered immediate and comprehensive on-board protection. The police can't be everywhere at once, but even when our driver called them, they didn't turn up – and this in Finsbury Park, which is hardly a hard-to-reach suburban backwater. More likely, the idea of dealing with racist intimidation on a bus was bumped right down the pecking order of priority calls, despite the number of people forced to suffer at the hands of the abuser.

Unreconstructed bigots will always roam the streets, but the rest of us deserve better than having to put up with their bilious assaults. Fear of fierce punishment is the best short-term deterrent, and would make anyone think twice before holding an entire bus hostage to their incendiary invective. Until that happens, the evidence of the perils of impunity will stack up on cameras, smartphones and YouTube to the collective shame of all Londoners.