So isn't this a mess? The results of the local and Euro elections won't be through in their entirety until Sunday, but we can say with some certainty that hundreds if not thousands of people who in all other respects one might consider decent have exercised their democratic vote to the benefit of a man apparently not averse to hobnobbing with the former Grand Wizard of the Ku Klux Klan. Communities in the north and a smattering in the south find themselves willing to make common cause with representatives of the BNP, a party that despises minorities and gay people and sees mixed-race children as a sickening affront to racial purity. The results we see so far don't meet the party's much trumpeted expectations, but it has claimed county council seats in Lancashire and Leicestershire – its first – and it polled strongly in another of its target counties, Essex. Nick Griffin hopes to be an MEP by Monday. Strange times. Worrying times. How did it come to this?

Is it that the Tories are to blame for encouraging xenophobia over Europe, as senior Labour backbencher Denis MacShane claimed on Comment is free last month? Certainly the Tory line on Europe and immigration more generally will not have helped. Is it Labour's fault, as Tory chairman Eric Pickles said earlier this week, getting his excuses in early? Well, certainly the bulk of the BNP's support occurs in areas that were once Labour strongholds. Labour ministers, watching traditional supporters drift away, have, at times, sought to entice it back with illiberal phraseology. Desperate measures, and not just desperate, but also futile.

And yet, in truth, what we are hearing from both sides is yah-boo stuff. It's irritating because they must know the problems involve both of them. This situation hasn't appeared from nowhere. It is the result of negligence. The architects of New Labour did not see the need to involve ordinary working people. They focused their attention on the limited numbers of seats and on the limited demographic they identified as necessary to get the party elected: Daily Mail readers and voters in the marginals. They cut the cord with the unions and the activist left, both of which were seen as a drag on their ambitions. But both provided grassroots links, connections with communities and foot soldiers. There was nothing to take up the slack.

They demoralised local government in the drive towards centralism, so too many good people opted out. Too many of those who replaced them enjoyed the status of office and the machinations of life in the town halls, but lost sight of the core requirement that they forge links with their constituents. Voters on estates I visited in Barking complained that they had not seen a councillor in 10 years. Into this vacuum came the BNP, telling lies as it suited them, exploiting fears about immigration, whipping up prejudices about crime and surfing a wave of Islamophobia.

But why was there a vacuum? Because Labour failed and, largely speaking, the Tories were no better. They too were chasing the middle class vote and the marginals and they were not about to worry themselves about areas they regarded as Labour fiefdoms, even those that had become failed states. They concentrated on the winnable seats. There is a logic to it. But the result of the decision by both parties to practise politics as the route to election victory rather as a tool for representing the broadest possible spread of communities left the door wide open for Griffin and his snake-oil salesmen.

Does this mean the BNPs supporters are all racist? Certainly the party at its core is racist. But for every ideologue follower there are others who have no wish to hobnob with types from the Ku Klux Klan. They just feel neglected and marginalised. The evidence so far is that where there is the option of a protest vote for a party other than the BNP, people are taking it. Some of their grievances are reasonable, some irrational, some based on falsehoods peddled deliberately, but in any event, before these can be addressed, the main parties are going to have to reconnect with those they have left behind. Not just by desperately spouting drivel such as "British jobs for British workers", but also by working much harder at local level. Mainstream politicians are going to have to win back their trust.

The BNP is a shambles with ludicrous policies and poisonous intent, but it thrives where its malignancy has been allowed to grow. It has not succeeded; rather the mainstream parties have failed.