Political correspondent Rowena Mason looks at how and why the party led by Nigel Farage has grown since the 2010 election

With his typical theatrical style, Nigel Farage kept a hall of Ukip activists on tenterhooks as he listed all the reasons why he should not return as party leader: his bad back, the exhausting schedule, the strain on his family. But he had changed his mind over a good English breakfast. “Do I have your support if I put my name forward?” he hollered, pantomine-style. The crowd roared in approval.

It was September 2010, months after the party gained 3% of the vote at the general election under Lord Pearson, the anti-Islam firebrand who led the party for just a year. Farage was toying with a backseat role, but believed that the coalition spelt trouble for the Conservatives and Lib Dems, that Labour had abandoned its working class base, and Ukip could “put the BNP out of business”.

Facebook Twitter Pinterest Ukip’s rise in the east of England: a world turned upside down – video

He then made this prescient statement: “We have got much to sort out in the party … Policy is part of the answer. But actually rolling our sleeves up and getting more professional about planning is the other part of it. The political opportunities are there, and greater than they ever were before. What we’ve got to do is get ourselves in gear.”

Four years on, as Farage prepares for another term in office, this appears to have happened at last. What was once a fringe anti-EU pressure group has the support of almost 20% of voters, millions of pounds, a target list of 25 seats and a high-profile Tory defector as its first elected MP.

On top of that, a relatively diverse band of frontbench spokespeople are selling policies on a broad range of issues, from the abolition of inheritance tax on the right, to the protection of the NHS against US private health firms on the left.

“Everybody can see the party is changing,” says Amjad Bashir, a British-Asian Ukip MEP and businessman elected in May. “I joined in 2012, and I’ve certainly noticed that we’ve got a lot more members … and because we’ve grown financially, we’re able to target good personnel.”

Facebook Twitter Pinterest Amhad Bashir campaigns in Keighley, West Yorkshire, ahead of the European elections in May. Photograph: Christopher Thomond/Guardian

The recruitment of big money has coincided with Ukip’s soaring fortunes, with hundreds of thousands rolling in, first from the Eurosceptic former Conservative Stuart Wheeler before the last election, followed by more ex-Tory donors, Paul Sykes and Arron Banks, over the last year.

One senior organiser, who has been close to Farage, points out Ukip received a major boost in EU staffing and resources when the party gained more MEPs in 2009. This came around the same time as a shift in thinking, he says.

“Years ago, it was two people in an office in Devon and two in the London press office. What changed was getting more MEPs in Brussels.

“Farage and the people around him began to realise there was the potential to turn this into a proper political party. The old Conservative voters were never going to give us everything we needed. We had to go after old working class Labour as well, who are some of the most patriotic people around.”

Ukip made an effort to get organised, modelling itself on the Liberal Democrats who targeted clusters of seats, and the right-of-centre, populist Canadian Reform party, which came from nowhere in 1993 to rout the country’s Conservatives.

At byelections, starting in the close-run Eastleigh contest of February 2013, the party rented visible premises, decked them in its yellow-and-purple branding, and used social media, mostly Facebook, to mobilise activists nationwide.

Facebook Twitter Pinterest The Ukip byelection office in Clacton, where Tory defector Douglas Carswell won the first seat for the party. Photograph: David Levene/Guardian

Buoyed by near-victory, Farage embarked on a “Common Sense” speaking tour, packing out English market town halls and working men’s clubs. He sang the same tune – about the costly EU, and identikit Westminster politicians – as he had done when he became an MEP in 1999. But more voters seemed to be listening.

Craig Mackinlay, a former Ukip leader standing for the Tories against Farage at the election, believes the party broke through at this point because it struck a chord on immigration at the right time in the political cycle – one of controversy about the accession of Romania and Bulgaria and outbreaks of right-wing populism across Europe.

“Things were all rather different in the old days of Ukip, which had more of an academic base,” Mackinlay says. “It was founded about the EU democratic deficit, now it seems to have descended into immigration and not much more. To an extent, that is a sadness. It seems to be a nerve that resonates with people and they [Ukip] are bashing that drum as hard as they possibly can.”

Mackinlay is still a firm Eurosceptic but would never return to the party, partly because of the excessive emphasis on immigration.

“If you ever go to a Ukip gathering, it looks like, dare I say this, the detritus there that have been everywhere else … Sometimes the talk of many of them [about immigration] is none too pleasant. You see that if you follow the tweets of the more virulent Ukip supporters,” he says.

Sexist, racist, and homophobic comments by some activists have certainly undermined the party’s efforts to appear more professional. But Farage has reacted with characteristic political acumen.

On the one hand, he has used it as an opportunity to paint Ukip as demonised by a media in hock to the politically correct establishment. On the other, it has provided an excuse to refresh his team – most notably, Godfrey Bloom, defence spokesman and MEP, who made ill-judged jokes about aid going to “bongo bongo land” and women being “sluts”.

Facebook Twitter Pinterest Godfrey Bloom was the centre of media attention over his racist and sexist remarks. Photograph: Luke MacGregor/Reuters

Janice Atkinson, an MEP and candidate in Folkestone, said the current top team was “not necessarily New Ukip but a more disciplined, professional and diverse set of candidates” who emerged to stand in the 2014 European elections.

“My MEP colleagues and I went through a rigorous interview process, better than the Tories’ which I also went through, and now that is showing in our spokesmen,” she says.

However, there is evidence this transformation has only gone so far. Of the more than 250 candidates picked to contest seats in 2015, Guardian research shows that nine in 10 of those are men, mostly older.

A number of controversial candidates have also been awarded plum seats. These include Victoria Ayling, a former Tory contesting the Labour-held marginal of Great Grimsby, who was caught on camera talking about wanting to “send the lot back” – although she claimed to have been talking about immigrants who were in the country illegally.

Farage himself has trodden a careful path on immigration in recent years, avoiding the anti-Islam rants of Pearson and arguing there should be more high-skilled immigrants from the Commonwealth. But he has always dropped enough bait to his core voters to make it clear it is OK to be concerned about a Romanian neighbour, that he is awkward about hearing foreign voices on the train, and more recently, that he would approve of HIV tests for new arrivals.

Such comments have the potential to create a clash with some of the party’s new recruits, including former Conservative Douglas Carswell, who said the HIV screening idea could not be taken seriously. Mark Reckless, the former Tory who looks likely to become Ukip’s second MP later this month, is concentrating his campaign in Rochester & Strood on the NHS and education, rather than immigration or Europe.

“Douglas and I are great believers in direct democracy,” he says. “I would like to see more emphasis not just on bringing back more power from Brussels, but from Westminster.”

Facebook Twitter Pinterest Mark Reckless talks to the media outside Rochester Castle the day after announcing his defection from the Conservatives to Ukip in September. Photograph: Matthew Lloyd/Getty Images

He also hints that power within the party could be shared out better: “A lot of issues get referred up to Nigel that I think in other parties would not pass the party leader’s desk. I think as the party gets bigger he needs to focus on some of the most important things and delegate more.”

But for others, the idea that Ukip could become a more professional political force is a cause for concern, particularly if some things that made its politicians more human are being watered down: the chaos, gaffes and shoestring amateurism.

“There is unease about what’s happening,” says one Ukip employee. “There is worry that the money men have taken over at HQ, with our Paul Sykes becoming head of policy. The people around Nigel are rich city people and they are able to influence policy in a way they couldn’t before. There is a worry we are becoming like all the others.”