What ungodly things must David Silvester think Ukip has done to deserve this ceaseless media shitstorm? Silvester is the party's Henley-on-Thames councillor whom it recently suspended for repeatedly asserting that the extreme winter weather is a punishment from God for legalising gay marriage. So what biblically prohibited acts must he think Nigel Farage has perpetrated that he is constantly tossed in a tempest of mockery and disdain? What sins has he committed to deserve such relentless tossing? What unholy transgression must Nigel have been guilty of to bring down these plagues of ridicule upon the organisation he's attempting to lead out of the wilderness?

Illustration by David Foldvari.

Did Silvester feel the hand of the Lord upon him as he wrote his nutty letter? Did he descend like an avenging angel into the BBC Radio Berkshire studio to reiterate his point? Was it God's work that Andrew Scott was doing on Monday when he allegedly attacked Farage with a placard bearing the words "Nasty Little Nigel"? Was Godfrey Bloom's assaulting a journalist with a brochure last autumn an example of the Lord moving in a mysterious way? Is Ukip, in fact, damned?

The Tories must be hoping so as Farage's party, in spite of these gaffes, seems likely to steal a significant chunk of their vote. Is "in spite of" the right phrase? Perhaps it should be "because of". After all, David Silvester used to be a Tory councillor. Conservative business minister Michael Fallon attempted to capitalise on Ukip embarrassment over Silvester's remarks by saying that "there clearly are one or two fruitcakes still around there". Fair point, but he's forgetting that many of these people used to be Tory fruitcakes. The Conservatives are losing the fruitcake vote.

Not that Ukip seems to value it. Like David Cameron, Farage is keen to shake off the "nasty party" image. For every Silvester or Bloom incident, he releases a compensating news story about how Ukip is becoming more credible and politically balanced. Last week, for example, he abandoned the party's entire 2010 manifesto in a TV interview and seeded in the media the concept of "New Ukip", in which all prospective candidates have to pass a day-long media skills and public speaking course. One of those who passed and is to be a by-election candidate, Ukip proudly announced, is a former Labour supporter! How normal does that sound?!

Ukip claims it's starting to attract women voters and the young. "Women are owning the show in Ukip right now," said a female activist. "The 'blokey men's club' perception of Ukip is incorrect". Lisa Duffy, the party director, said they're "attracting more of the student vote… When we come… they may not be Ukip supporters, but by the time we leave that changes." And Farage has even spoken in favour of Britain taking in Syrian refugees.

So Ukip is following where Blair and Cameron have gone before: it's making a play for the middle ground. It's saying: "We're just decent normal people who represent the values of other decent normal people." "Hard working families", "alarm clock Britain", "the squeezed middle" – like Satan, they have many names, and all politicians want to appeal to them. "We're for you, you noble struggling put-upon, good-hearted societal contributors! Vote for us, we've got your back!" say all the parties.

Which begs the question: who is there for the nasty to vote for? If the Tories aren't the nasty party any more, and now Ukip isn't either; and if Labour is no longer the party of benefit cheats and power-hungry trade unions, but is dedicated to helping ordinary working families, whom the Tories are also dedicated to helping rather than fat-cat bankers and landed aristocrats; and if Ukip is no friend of racists, homophobes, sexists and Christian fundamentalists but welcomes women, students and Syrians; and everyone had forgotten the BNP even existed till Nick Griffin went bankrupt to give us a New Year laugh; then who will electorally benefit from the nasty demographic?

It's not as if there aren't plenty of nasty people in Britain, or that they can't vote. I suppose some are in prison – and you'd hope that the percentage of prisoners who are nasty was higher than that of the overall population. But there are still millions of law-abiding enfranchised unpleasant citizens to whom the political mainstream and Ukip are pointedly not reaching out.

One of the problems facing the nasty is that they don't agree about anything – which is probably why Ukip has proved so ungovernable. Some are rich and believe they should pay no tax, others fiddle their benefits. Some hate foreigners, some hate black people, some hate white people, some hate gay people, some hate women, some hate men. They're all out there, bickering about what's horrible, motivated to exercise the franchise, but no politicians want to know. Just last week, Tory Treasury minister Nicky Morgan said: "If we talk about what we hate all the time, we're not talking about what we like." But that's not true: what if what we like is hatred? There's a significant bile-spouting section of the community – just look online.

I almost admired our political class for ignoring those with reprehensible views until I asked myself "Who will those who agree with what David Silvester says about gay people vote for now?" and I realised the answer is almost certainly Ukip. Even though Ukip has disassociated itself from that opinion, the very fact that it had to do so suggests it's a party that homophobes are drawn to.

Similarly benefit cheats tend to vote Labour. Ed Miliband has worked hard to distance Labour from the "something for nothing" culture, but more significant is the fact that he felt the need to do so – a Tory leader wouldn't. But most tax-avoiders will vote Conservative, despite government rhetoric. You can detect a party's unsavoury hinterland not from the people it courts but from the ones it disowns – whose votes it will probably still get.

So, in light of Ukip's rise, it may not be entirely bad news for the Tories that Aidan Burley is back in the papers. He's the hotly tipped Conservative MP who organised a stag do in France at which the groom, Mark Fournier, was wearing an SS uniform (which it later transpired Burley had hired) and a "Nazi-themed" toast was drunk. Last week Fournier was fined €1,500 by a French court for "wearing a uniform or insignia of an organisation guilty of crime against humanity".

No party wants to be seen to appeal to the sort of person to whom that sort of thing appeals. But the central office buffet is running low on fruitcake.