Last Thursday the British public queued at their polling stations to quietly and politely lay three ticking bombs under British politics. If we don't hear the tick-tock and steadily defuse the voters' anger, the eventual blasts will damage Britain for decades to come.

The first shelf-full of Semtex was aimed at the Prime Minister. I hate talking about politics in terms of personalities, and I find the vindictive tone towards Gordon Brown – booing him at a D-Day commemoration? – unpleasant. But we can't live in a fantasy world. Brown has turned out to be electoral kryptonite. Labour has just taken its lowest share of the vote since 1910, before the First World War – and if it keeps on marching in the same jerky formation, it is heading for a political Somme.

The reasons why the Brown stuff has hit the fan have to be understood properly. The attempts to take Brown down have come almost exclusively from the Blairite wing of the Labour Party – people like Stephen Byers, Hazel Blears and Charles Clarke. They have always thought Brown was too left-wing, and now grasp for his few tiny millimetre-shuffles towards social democracy as explanation for his failure.

Download the new Independent Premium app Sharing the full story, not just the headlines

But this is surreal. Gordon Brown has failed because he has been paralysed, unable to take any substantial decisions at all – except to keep drifting in a Blairite direction. With the honourable exception of using the state to stop the banks collapsing, he has carried on with hardline Blairism: building more airports, trying to part-privatise the Post Office, and apologising profusely to millionaires for his meagre tax rise, even though 68 per cent of the public support it.

In reality, he has failed because of a double-whammy: he has continued with lousy and unpopular right-wing policies, and he sells them appallingly. If he remains as Labour leader, he will hand the country to David Cameron, who will dismantle the few good left-wing policies that snuck through New Labour – tax credits, SureStart, Educational Maintenance Allowances.

The Independent's poll shows that Alan Johnson can stop the haemorraghing and confine Cameron to a hung parliament, where he will be able to do far less damage. Labour would need more commitments from Johnson that he will respond to the recession-heavy mood for greater social democracy – but it would be an act of political self-harm to stay with Brown. It is sad for his political career to end like this, but it will be much sadder to be poor in a Cameron-led Britain. In the name of Gord, go.

The second bomb came crashing in from the extreme right – the BNP – whic now has two MEPs, both with records of extreme bigotry. Nick Griffin has bragged about how much he learned from Adolf Hitler's Mein Kampf, while as a young man Andrew Brons joined the National Socialist Movement, set up on Hitler's birthday as a tribute to him.

But it is not the case that 10 per cent of people in Yorkshire are sympathetic to Holocaust-denying lunatics. No: they were overwhelmingly skint young white men who would, a generation ago, have formed the Labour core vote. They are angry about low wages and chronic shortage of housing – and simply telling them they are bigots won't get us very far.

Any conversation with BNP voters has to begin by agreeing that they are right to be angry about both subjects. There is a housing scandal in Britain today. In the 1980s, the revenues from council house sales were squandered by Margaret Thatcher on tax cuts for the rich, instead of being used to build more social housing. Labour allowed social housing construction to fall even further. We now have a housing drought, leaving hundreds of thousands of people stuck in cramped, damp homes. Similarly, our minimum wage is one of the lowest in the developed world. Tax credits are good, but today they only go to people with families: the rest watch their wages sink.

Only once this is agreed should the conversation move on to the fact there is a more effective and more deserving outlet for their rage than other poor people with different pigmentation. The white working class has a shared interest with black and Asian people in demanding higher taxes on the wealthy – and less squandering of cash on pointless projects like Trident and ID. cards – to lift up everyone stuck at the bottom. But what mainstream party has advocated that for years? In the absence of any socialism, we will see its antithesis, National Socialism, rise as a nasty intimidating fringe. If we want to choke this off, we need to deal with the real issues it feeds on.

The third bomb is the rising rejection of the European Union. We have now reached a point where Britain's governing party has been beaten by an organisation whose sole purpose is to yank us out of the EU. This undercurrent is tugging at the entire political system: the Tories in the European Parliament are now withdrawing from their alliance with Angela Merkel, Nicolas Sarkozy and the rest of the European centre-right, and cobbling together a coalition of Polish gay-bashers, Czech global warming deniers, and assorted Europe-hating loons to sit with. Cameron's Tories would form the most anti-European government in living memory: they even voted against European co-operation to track down child molesters.

How long can we spit at the 450-million-strong trading block on our borders, with which we do 60 percent of our business, before it has consequences? How long can we try to kick out the foundations of unprecedented peace in Europe before it begins to crack?

This can't go on. The Liberal Democrats – a lonely, brave British voice in defence of Europe – have been arguing for years that we need a rerun of the 1975 referendum: should we stay or should we go? The arguments for Europe – and the real cost of leaving – would be drawn into the open. The case would become clear at last.

Since the majority of our trade is with the EU, after withdrawing we would have to abide by almost all its rules anyway to be allowed to sell to them – but we wouldn't have any influence on drawing them up. Think of it as the United Kingdom Isolation Party, where we won't even be on the sidelines; we'd be outside the stadium, on an empty street.

So we would gain little, but we would suffer horrible self-inflicted wounds. Three million jobs would melt away to a Europe that would now be wrapped away behind tariff walls. The millions of Brits living elsewhere in the EU – one million in Spain alone – would be left stranded, and have to come home, or apply for immigration rights they were no longer entitled to.

Our ability to shape the future of the world, especially on global warming and foreign affairs, would be dramatically diminished. Our ability to reform the very real flaws within the EU would be gone. And we would have helped to bring down an extraordinary political project that shows the world that even the most bloody and war-ravaged continent can pool its sovereignty and live together in peace.

The electorate just booted Brown, brown-shirt nostalgists and browning-off Europe to the top of Britain's agenda. The ticking will only get louder if we try to brush these bombs under the Westminster carpet – and carry on as if the people have not spoken.

j.hari@independent.co.uk