“On your bike, Nigel,” the hecklers mocked. It was not meant to end this way for the Ukip leader. Standing in South Thanet, Nigel Farage expected finally to become an MP ­after six failed attempts.

When his rejection by the electorate was made official, he gave a terse speech, stressing his “relief” at the result. He then marched out of the count before the other losing candidates had made their speeches. Defeat with dignity this was not.

South Thanet, a sprawling coastal constituency in Kent with a large population of relatively low-skilled workers and high anxiety about immigration (although white Britons account for 90 per cent of its population), had seemed an ideal fit for Ukip. The abandoned shops in the centre of Ramsgate, the heart of the seat, give the impression of a town whose people have been left behind, even if Broadstairs and Sandwich, elsewhere in the constituency, are more affluent. A series of financial scandals at the local council rendered Thanet more susceptible to Ukip’s brand of anti-politics.

So, when Farage announced last year that he would stand in South Thanet, the popular assumption, underpinned by constituency polling, was that he would win. Indeed, two months before the election that was the view shared by the Conservatives. Sensing that the local branch was struggling, they sent Nick Timothy, chief of staff to Theresa May, who had worked on three by-election victories before the 2010 election, to live and work full-time in Ramsgate. The Tories sensed that to beat Farage they first needed to beat Labour, to prove that only they could stop Ukip winning. “The person that was going to win the election was the person who could unite the anti-Farage vote,” said Will Scobie, Labour’s losing candidate.

In their campaign leaflets the Tories, who gained the seat in 2010 from Labour, described the contest as a two-way battle between themselves and Ukip, and cultivated a few prominent local Labour figures who advocated voting tactically to stop Farage. “We were seen as the lesser of two evils with people who identified as centre-left,” a Tory campaign insider told me.

In the final push, the Conservatives ­focused on two main strategies. Farage was branded the “absentee candidate”, who didn’t care about the area and wouldn’t work hard as a local MP. The Tories championed the local links of their candidate, Craig Mackinlay, who walked 90 miles in the constituency in the last week before the vote. And the Tories vowed not to attempt to “out-Ukip” Farage. Though Mackinlay was a former deputy leader of Ukip and thus well disposed to winning over “soft Kippers” on the doorstep, the party avoided mention of Europe or immigration in its literature, fearing this would legitimise Ukip’s policy platform while alienating tactical voters. The focus was on the economy, jobs and local issues such as regenerating Ramsgate.

The Tories were also helped by fears of the SNP exerting power in a coalition, which resonated with patriotic voters inclined to vote Ukip. This is one explanation why many voters split their ticket on election day: although Farage polled only 32 per cent, compared to Mackinlay’s 38 per cent (and Scobie’s 24 per cent), locally Ukip won a majority on Thanet District Council.

Farage’s polarising nature may have repelled as many as he attracted to him. As Ukip sent money, foot soldiers and expertise to South Thanet, efforts mounted to prevent Farage from winning. Stand Up to Ukip hosted numerous events in the seat and anti-Ukip badges were visible on polling day in Ramsgate. There is an important lesson for the imminent EU referendum, as some who advocate leaving concede: Farage risks being as toxic as Nick Clegg was in the AV referendum four years ago.

Like Clegg on tuition fees, Farage now risks being mocked for a broken promise. “It is frankly just not credible for me to continue to lead the party without a Westminster seat,” he wrote in his memoir The Purple Revolution, published two months ago. Farage did resign but 72 hours later gave in to what Ukip’s chairman called “overwhelming evidence that the Ukip membership did not want Nigel to go”.

South Thanet wasn’t the only Ukip disappointment on election night. The party failed in other target seats such as Thurrock, Castle Point, and Boston and Skegness, leaving the Tory defector Douglas Carswell as its sole MP. The defeat of Mark Reckless makes future defections to Ukip less likely.

As Ukip painfully learned, first-past-the-post is ruthless to insurgent parties. Yet 3.8 million people voted for it on 7 May, 2.4 million more than backed the SNP. Ukip came

second in 120 seats and hurt Labour at least as much as the Tories. Across the north-east, north-west and Yorkshire, once considered Labour’s unbreachable northern heartlands, Ukip averaged 12.5 per cent in seats that were Tory-held before the election but 16.9 per cent in Labour constituencies: indeed, it was the Ukip surge that was decisive in defeating Ed Balls. Core Labour voters have proved susceptible to Ukip’s message, giving the party a base from which to launch a renewed assault on Labour at the next election: the “2020 strategy”.

It might not be as simple as that. The ­aftermath of the EU referendum will pose a fundamental challenge to the party. Tensions will simmer between blue and red Ukip – those who believe its future lies in peeling off shire Tories and those more focused on the northern working class. And eventually Ukip will need to move on from Farage who, for all that he has brought to the party, is an unparalleled loser in Westminster elections.