At the Methodist Central Hall in Westminster, Nigel Farage stands at a podium waiting to address the Homes for Britain rally. Beneath its vast dome he looks small, depleted. The arms hang limply at his side; his forehead shines with perspiration. Perhaps he is wondering: will this be a tough crowd, or a sympathetic one?

Jonathan Dimbleby, the rally’s compere, has just reminded the audience it must treat each speaker with equal courtesy; it’s clear that he fears Farage will be booed.

Nevertheless, this could go either way. Out here – I’m in the Gods, fighting the tears brought on by the appearance of a choir whose members have all experienced homelessness – are 2,300 people from 300 organisations. Many, it’s true, are liberal-minded campaigners for social housing. But among their number are also hundreds of developers and landlords. They may look on the leader of Ukip more kindly, particularly were he to announce that he and his libertarian band of brothers are set to give up on the green belt.

This is the most eclectic political party in Britain. We had goths, we had people in turbans

The short film we’ve been watching ends, and Farage begins. He has eight minutes to set out his stall. He kicks off with some personal stuff. He is, he tells us, a sentimentalist. He grew up in Kent, not far from where Darwin wrote On The Origin of Species, and he would not see its hedgerows destroyed for all the world. So the green belt must remain. However, unlike everyone else, he is not going to drone on about housing supply: it’s demand that interests him. Though Ukip believes in the idea of a “brownfield revolution” – it would remove stamp duty and VAT from all new builds on brownfield sites – the essence of his message is that the housing crisis can only be solved if net migration is curbed. Only one party has the courage to do this: his own.

In the hall, the atmosphere shifts. The rally’s good humour has dimmed. Farage, though, seems to grow more confident, drawing strength both from his audience’s disdain and from the fact that the finishing line is now in sight. His face might be slick with sweat, but by the time he stops speaking his manner is shot through with the tinny bravado that comes of being flooded with relief (only rarely does Farage speak from notes; mostly he prefers to wing it).

Facebook Twitter Pinterest Constituent parts: in Thanet South, where Farage hopes voters will make him an MP. He has said he will stand down as Ukip leader if he doesn’t win. Photograph: Gareth Fuller/PA

From my seat, I consider my next move. His people have an aversion to answering my calls. They seem not to want me to see him in any other context than one they’re able tightly to control. But the Methodist Central Hall stands next to one of his favourite pubs, the Westminster Arms; it’s hard to believe he’ll slip away without scooting over there for a pint of Spitfire or Whitstable Bay. As he exits the stage I rush out, determined to catch him.

“Ha ha!” booms Farage delightedly when, the next morning, I confess that I waited in vain for him outside the Westminster Arms. “Two years ago, during Ukip’s conference, that’s precisely what I did. I slipped out for a pint. But alas, I had to get on yesterday.” He emits a hammy sigh. “It’s a difficult time. I’ve got to support the party in other parts of the country, and I’m a member of the European parliament, and I’m fighting in South Thanet, a seat that isn’t safe like those of the other leaders.” Is he knackered? “Yes. But then I have been for years. Ukip really started to motor at the end of 2011, and it has been nonstop ever since.”

It is budget day. Across Lambeth Bridge, George Osborne will shortly begin shooting Labour foxes. Yet here is Farage, marooned at the publisher of The Purple Revolution, his new book, giving interviews. It goes without saying that he’d rather be in front of the television, fag in hand. In this cramped corner office, moreover, it is greenhouse hot, and though this seems not to bother his “senior adviser” Raheem Kassam – a man who wears his overcoat as if it were armour – Farage is distinctly pink of face. It’s possible, too, that he has a headache. Last night he launched The Purple Revolution at – yes – a Westminster pub. Is he hungover? “It was a great evening,” he says, ignoring the question. “Normally people hang around for the speech, then they go. But yesterday everyone stayed on.” Soon after this, he turns to Kassam and croaks: “Could I possibly have some more water, Raheem?”

Even by his standards – Farage goes to bed at midnight and rises at the crack of dawn, seven days a week – the last few days have been relentless. The newspaper serialisation of his book threw up several controversies, plus there was the fallout from a Channel 4 documentary in which Farage told Trevor Phillips, former chair of the Equality and Human Rights Commission, that UK race discrimination laws should be abolished. Debate, horrified or otherwise, about Ukip policy in the matter of the children of immigrants – Farage has suggested they should not have access to the state education system for five years – rumbled on, as did his latest comments about Muslims (he believes they want to form “a fifth column and kill us”). There has been no escape – for him or for us. Even as we talk, another crisis is brewing. He does not yet know it, but Janice Atkinson, the Ukip MEP who was to fight Folkestone and Hythe at the election, will shortly be suspended from the party for “financial irregularities”; Stephen Howd, the party’s candidate in Scunthorpe, will stand down while an alleged incident at his workplace is investigated; and Jonathan Stanley, who was to stand in Westmorland and Lonsdale, will resign, having accused Ukip of racism and bullying.

Facebook Twitter Pinterest Village protest: demonstrators surround Farage’s car last Sunday. Photograph: Levi Hinds/PA

The line people like to take about Farage, particularly those who disagree with him, is that he is a clever, canny operator, one we underestimate at our peril. Certainly Nick Clegg seemed vastly to have underestimated him when he challenged the Ukip leader to a televised head-to-head on Europe last year (Farage was judged the clear winner). But I’m not so sure about this. Even taking into account his tiredness and the fact that he clearly doesn’t believe I’m worth an hour of his time, his inability to sustain an argument is astonishing. Little of what he says bears even the slightest scrutiny, for all that he interrupts with alacrity. He reminds me powerfully of Zippy, the ridiculous puppet star of the children’s programme Rainbow, and thanks to this I soon morph into George, Zippy’s hippopotamus companion.

George, you will recall, was gentle, mild-mannered and lightly ironic of tone. Even so, by allowing Zippy simply to burble on, he usually won in the end.

The only people who think we’re racist are white. The black community is incredibly friendly

Did the fallout from the conversation he had with Trevor Phillips about discrimination and the law take him by surprise? “Yes, I spat out my tea,” he splutters. “I’d forgotten I’d given that interview. But the argument I was making was not: let’s abolish these laws. The argument is: there are 5 million small businesses in this country, and 250,000 unemployed youngsters, and I want to give both a better chance. I think there is a strong argument for relaxing them.” How would this help? “There should be a strong presumption in favour of nationality. I mean: in favour of UK passports.” But what about black Britons? They might find themselves on the receiving end of discrimination, British passports or not. “The example I use is: Poland.” So you want employers to choose Britons over Poles? “Yes.” And beyond that they can pick their staff according to their own whims and prejudices? “Oh, this is remarkably uncontroversial,” he says.

What about excluding the children of migrants from the education system?

Apparently this is still up for debate. “I mean, if someone’s coming in and doing a skilled job that’s not highly paid, clearly this may be unfair. But if they’re working for Goldman Sachs, should we provide their children with an education?” Is he serious? How many highly paid foreign bankers does he know with children at British state schools? His voice rising in exasperation, he makes one of his habitual rhetorical leaps. “The question is: do we think it’s reasonable to have uncontrolled numbers of migrants coming into England and bringing their families with them? We’re 250,000 primary school places short by 2019.”

Looking at his weary face, clammy and puce, it’s impossible not to wonder about his promise to stand down as Ukip leader should he fail to win Thanet South at the general election. Before today, I took this announcement as a sign that he believes those polls which put him ahead of the Tories, 11 points clear in one case. But perhaps a part of him is looking for an excuse to quit. There is gossip about his ill health (though when I ask if he is in pain – his back has hurt ever since the famous plane crash of 2010 – he says not: “It would hurt if I’d been in the car for eight hours, as I often am, but today it’s OK.”) So which is it? Is he secretly hoping to fail? “I said the same last year about the European elections: that if we didn’t win I’d stand down. You could say that I keep playing double or quits.” How would he feel if his political career came to an end in five weeks’ time? “It would be wonderful! I’d have my life back. I could get a proper job.” A pause, during which he grins at me determinedly. “I’m being a bit flippant,” he says when I don’t grin back.

Facebook Twitter Pinterest ‘I work a bloody long day, and if I can go out and have a glass of wine and a good laugh, then I will’: Nigel Farage. Photograph: Pål Hansen for the Observer

It used to be Farage’s proudest boast that it was impossible to professionalise the Kippers, but as the election approaches, professionalism is, he must admit, at something of a premium. “It’s the biggest dilemma for me, isn’t it? I want people to speak freely, but I’ve got to create a party that succeeds.” Does he believe Ukip still has a problem with what David Cameron called fruitcakes, nutters and closet racists? Watching a recent documentary about its activities in Thanet, it seemed that it might. “Yes. Yes, we do. But we also get more scrutiny than other parties. Two weeks ago, a [former] Labour party agent was sent to prison for sex crimes. Did you read about that? No, you didn’t. But my people get on the front page for saying something vile at 11 o’clock at night on Facebook. We need a sense of perspective. This is what the establishment is trying to do to us. They’ve turned a percentage of the population into actively hating us.”

What, then, does he make of the rank and file, five weeks before a general election? Is he confident, as they prepare to launch themselves on our doorsteps? He waves an arm. “I looked at them at Margate [during Ukip’s spring conference], and I thought: crikey, this is the most eclectic political party in Britain.” Could he describe them? “We had goths, we had people in turbans…” His voice trails off. And does he think this, er, diversity reflects the success of Ukip policies, or are these merely people who have nowhere else to go? For the first time today, he sounds almost enthusiastic. “No!” he yelps. “These people actually believe in it!”

You don’t get it, do you? Seriously, it worries me that so few of you understand what we’re saying

The Purple Revolution is subtitled The Year That Changed Everything – a reference to 2014, in which Ukip gained its first MPs following the defection of Douglas Carswell and Mark Reckless from the Conservatives, and came within a whisker of taking Heywood and Middleton from Labour. (“Another 30 votes and Ed would have gone,” says Farage. “I think they would have booted him out.”) For Ukip’s leader, these events were thrilling: after the count in Clacton, Carswell’s constituency, his exhilaration was so great he leapt over the bar at the pub where the Kippers had gathered to celebrate. But on the page he struggles to convince the reader of their wider significance. Partly this may be to do with hindsight: Clacton and Rochester seem now to represent Ukip’s high-water mark. Mostly, however, it’s the result of Farage’s Pooter-ish idiom, in which the banal and the overblown collide to unintentionally comic effect. “In this proud Anglo-Saxon town, something amazing had happened,” he writes of Clacton. “The Anglo-Saxons had a view of life – that it is a bird that flies from the darkness into the warmth and community of the mead hall, then back into the blackness. That night in Clacton… Ukip was finally alive. We were flying into the mead hall.”

The book takes in his love for his “political doppelganger” Rand Paul, the Republican senator and Tea Party supporter, and his loathing of, among others, Baroness Ashton, the EU’s former High Representative for Foreign Affairs (“She was a woman who had married well. That was it. No talent, no ability, no skills, no experience.”) But he also devotes several pages to the virtues of his beloved school, Dulwich College; to his passion for Keith Joseph, the Thatcherite who first stirred his political instincts; to his time as a metals trader in the City; to the testicular cancer he suffered in his 20s; and, above all, to his fondness for a long lunch.

Facebook Twitter Pinterest ‘Ukip really started to motor at the end of 2011, and it has been nonstop ever since’: Farage in the European parliament, after being made an MEP in 1999. Photograph: Mark Lloyd/Daily Mail

In his City days, the Farage lunch was known as the “12 till 12”, a tradition he is unwilling to give up without a fight. In America to meet Rand Paul, he is appalled to be told, come one o’clock, that sandwiches have been ordered: “My feelings fell somewhere between panic and indignation,” he writes. In the end he, Raheem and another colleague, Matt, had no choice but to bolt to a steakhouse. “Matt had a 27oz burger with a stack of onion rings – halfway through he began to sweat profusely. It was a corker lunch.”

Though Farage is much given to pointing out the corruption and laziness he believes is widespread in politics, he doesn’t think twice about providing such insights. “I work a bloody long day, and if I can go out and have a glass of wine and a good laugh, then I will.” His rejection of the new puritanism, he says, marks him out from most politicians, as does his absolute refusal to wheel out his family (he has four children, grown-up sons from his first marriage, and two teenage daughters by his German second wife, Kirsten). “What’s my kitchen like? I’m not telling you. I won’t even let you through the front gate. Whatever my faults, I have some principles. It’s selfish, politics, and everyone around you suffers because of it.” Does his family suffer? “Of course. The lack of a normal family life, because there can’t be one and… even going into the shops and putting your credit card on the counter is potentially a danger [for our family], because our name is very unusual.”

Do his sons share his views? “I’ve no idea, and if I did, I wouldn’t tell you.” It’s fascinating that he is married to a German. “Why? I’ve no idea what you’re talking about.” He is cross. “You don’t get it, do you? Seriously, it worries me that so few of you understand what we’re saying.” Oh, I understand. But what I want to know is: has Kirsten ever asked him to tone down his xenophobia? “She is the type of migrant who would pass the Australian-style points system we want to bring in,” he says, which isn’t really much of an answer at all.

I want people to speak freely, but I’ve got to create a party that succeeds

Does it matter to him what people think? Does it bother him, for instance, that I consider some of his views racist? “This is interesting. The only people who think we’re racist are white. The black community is incredibly friendly. It bothers me, yes, that you’ve been driven to think that, because it’s unjust. Clearly you don’t understand me. I can tell you don’t get where I am coming from.”

Actually, I say, I think he’s the one who doesn’t understand me. I grew up with men like him; they patronised the Sheffield pub where I worked as a barmaid, and I thought about them at the age of 19 what I think about him now, which is that a lot of their political opinions were born of insecurity. An indignant snort. “If I was insecure, I don’t think I would have given up a good career to go into politics, do you? Would I have joined a party with 250 members and no prospects?” Perhaps, I say. After all, isn’t it better to be a big fish in a small pond?

His new book skates over his childhood, but ask Farage about it, and suddenly he is unable to meet your eye. He was born in Downe, near Sevenoaks, the Kent village where he still lives (and where, last Sunday, his lunch was ruined when protestors invaded the pub where he and his family were eating). His father worked in the City; his mother was a housewife. They divorced when he was six.

“It was an up-and-down childhood,” he says, “which in those days was unusual. I can distinctly remember being the only boy in my class whose parents had separated.” Did his father leave for another woman? “He had his own problems.” (His father, it has been reported, was an alcoholic.) Is this why Farage is so attached to the idea of school, because it provided the stability he was missing at home? “No, no, no! I didn’t have a rotten time of it. I lived in a nice village on the Kentish downs. I had tremendous freedom. My brother and I would go out for the day, tuppence in our pockets, to collect things.” Like what? “Nuts, mushrooms, you name it…” He seems to be scratching round for something. “And, of course, we’d shoot. Kids had air rifles in those days.” Did his mother remarry? “Yes.” Did she work? “Not until much later.” He coughs, awkwardly. “You may have seen charity calendars of her, half-naked. She’s quite a character.” No, I say, I haven’t. He looks disbelieving.

Facebook Twitter Pinterest ‘I don’t think I’ve changed at all’: in the 2013 Eastleigh by-election campaign. Photograph: Matt Cardy/Getty Images

He loved Dulwich College, and no wonder; it’s a good school, though he did not distinguish himself at A-level (a teacher told him he should be an auctioneer, which sounds about right). However, some of his claims about it are on the outlandish side, particularly his insistence that thanks to its amazing social mix, he is more able to talk to working-class people than most other politicians. “Well, 50% of boys were on assisted places,” he says. “One had a father who was a coal merchant.” Ah, yes. Farage is obsessed with this fellow, a man who, according to his book, hailed from Penge; in his imagination, coal merchants inhabit an underclass all of their own.

But then, personal experience is all to Farage; it’s on such anecdotes that his policies are based. For instance: the cancer that the NHS failed to diagnose correctly when, almost 30 years ago, he presented himself with a testicle as “large as a lemon, and rock hard”. Thanks to this, and the treatment he received for the injuries he sustained in the plane crash, it should be perfectly obvious that he is “better qualified to criticise and defend the NHS than most politicians”. I tell him that although I was treated for a skin-cancer scare within three weeks, I don’t sail along thinking there is no need for NHS reform. So why should it work the other round? Why should a single bad experience dictate his view of a vast system? It’s foolish to extrapolate from isolated incidents. But he seems not to have an answer to this. “Every family has three or four good stories and one bad one,” he mutters. OK then. Let’s extend this storytime policy initiative to immigration. What does he say to those whose experience has taught them to favour hard-working Polish builders over their British counterparts? “Blair started this!” he shouts. “The metropolitan dialogue that says British builders are useless, lazy, fat and crooked!”

It is impossible to believe that Farage is only a few years older than me. He seems to belong to another generation entirely, another world, even. Was he always like this? He smiles proudly. “I don’t think I’ve changed at all.” Did he like punk? “Nah!” What music did he like? “None.” What does he like now? Is it Elgar’s Nimrod all the way? “I don’t listen to music, I don’t watch television, I don’t read.” So he’s exactly the kind of political robot he purports to despise? “No. My hobbies haven’t gone away. When I’m finished with politics, I’ll have a richer life. I’d like to go to the theatre.” To see what? “Everything from Chekhov to… contemporary stuff.”

In the moments before he leaves – a shiny black four-wheel drive will whisk him off, leaving me on the pavement listening to the rush of the traffic – I try to get him to talk about his fierce attachment to the past. At first he resists. “I’ll tell you what’s in its death throes, that’s the European parliament,” he says, launching the case for his own modernism. But in the end he can’t resist. The lure of a prelapsarian Britain conjured from – I’m only guessing – boys’ annuals and Ealing comedies is too strong. Yes, he would like to see the return of matrons in hospitals and grammar schools. Terraces at football grounds? He shakes his head. Not his game. An overturn of the smoking ban? In pubs with well-ventilated separate rooms, certainly. What about greyhound racing? Why doesn’t Ukip support this beleaguered British sport? At last, his eyes light up. “Now that would be an idea. A night at the dogs.” The laugh is loud, but something in his distant gaze tells me that he’s only half joking.

The Purple Revolution (Biteback Publishing, £9.99) is out now

Follow the Observer Magazine on Twitter @ObsMagazine