What does Nigel Farage know? What does any successful politician know? What did Tony Blair know that Ed Miliband did not? What does Jeremy Corbyn know that his detractors in the Parliamentary Labour Party do not?

In 2009, Michael Ignatieff, a cosmopolitan intellectual and former Harvard professor, became the unlikely leader of the Liberal Party of Canada. As he began the slog towards the Canadian federal election, from which he was initially expected to emerge as prime minister, Ignatieff was tormented by his inadequacies. High intelligence, deep, immersive reading and considerable literary and philosophical sophistication – he was the authorised biographer of Isaiah Berlin and a former Booker Prize-shortlisted novelist – were, he discovered, no guarantees for a career in politics or for winning a national election.

“I’ve spent my life as a writer, but you have no idea of the effect of words until you become a politician,” Ignatieff told his old friend, the New Yorker writer Adam Gopnik. “One word or participle in the wrong place and you can spend weeks apologising and explaining.”

It was as if he was already exhausted by the demands of high politics: “This is by a very long shot harder than being a professor at Harvard, harder than being a freelance writer, harder than anything I’ve ever done – in terms of its mental demands, its spiritual demands and its emotional demands.”

Ignatieff envied successful politicians, serial winners such as Blair and Bill Clinton. He knew they knew something he did not. But what did they know? What is it that a great politician knows, he kept asking himself. “The great ones have a skill that is just jaw-dropping, and I’m trying to learn that.”

Ignatieff never discovered the answer to his question or learned the required skills. Unlike Barack Obama, who was also professorial in demeanour, he had no gift for popular communication. Nor was he adept at the game of politics – or perhaps ruthless and fearless enough, though he was more than ambitious enough. He was routed in the 2011 election by the conservative Stephen Harper, even losing his own seat. Soon afterwards he retired from politics and retreated from Canada, humiliated and humbled by defeat, but wiser.

I was reminded of Ignatieff and his pertinent question – what does a successful politician know? – when I met Nigel Farage one recent morning at the offices of Leave.EU in Westminster. The previous evening, when I texted a friend to postpone our meeting because I was seeing Farage, he replied: “Why are you seeing him? I despise him.” This is not an isolated view, of course.

Farage, who now has his own talk radio show on LBC, is widely despised – not least because of his antics during the referendum campaign and his post-Brexit embrace of alt-right movements in America and Europe. He is despised not only by liberals and Remainers: mainstream Conservatives and many prominent Brexiteers, such as the MEP Daniel Hannan, are appalled by him and his closest associates at Leave.EU.

***

The Leave.EU offices are subdued and tatty – they have the atmosphere of a poorly resourced magazine or newspaper office the morning after press day – but at least there is an outside terrace, which allowed Farage to slip out for a cigarette on a cold, bright morning. The television was on in his office and it burbled away as we talked. A packet of Benson & Hedges cigarettes was on the desk and on the bookshelf nearby was a paperback copy of the Cambridge historian Robert Tombs’s great book The English and Their History. One of his aides brought him a coffee from Pret A Manger – “I can’t drink that instant stuff” – then Farage settled down, preferring initially to discuss the Ashes cricket series in Australia: a few overs of gentle conversational looseners before the pace quickened.

Farage was in a reflective mood. A former City broker in the metals market who was educated privately at Dulwich College in London (a school today popular with Russian oligarchs), he still sees himself as an anti-system radical, who occupies a space beyond left and right: he told me once that his hero was John Wilkes, the 18th-century parliamentary agitator and pamphleteer.

Unlike my friend, I do not despise Farage, even though I deplore much of what he says. What does he know? That’s what interests me. It’s not enough to condemn one’s opponents: it’s harder, yet more fruitful, to attempt to understand and explain the forces and individuals shaping the history of our era.

Speaking on 14 November in the Commons, Ken Clarke, perhaps the last true Tory parliamentary Europhile, called Farage the “most successful politician of my generation”. It’s hard to disagree, though he tried and failed seven times to become an MP.

More than any other politician – more than the cranks and headbangers on the Tory fringes – he created the conditions for Brexit, and we are living with the consequences. Through sheer force of will, charisma and a kind of relentless monomania, Farage transformed what was once a fringe cause into a national movement (the “people’s army” is what he calls his followers). He galvanised the Eurosceptics in the Conservative Party and harried David Cameron, who in 2006 dismissed Ukip supporters as “fruitcakes, loonies and closet racists, mostly”.

For all of Farage’s success at ventriloquising the sentiment of a large section of the population, his behaviour has often been contemptible: never more so than when, in the final week of the referendum campaign, he launched the anti-immigrant “breaking point” poster depicting a column of Muslim Syrian refugees in the Balkans, the wretched of the Earth. Farage deliberately conflated legitimate economic migration with the refugee crisis and illegal immigration: even the former Ukip MP Douglas Carswell called the poster “morally indefensible”.

Farage remains unapologetic. “Jacob [Rees-Mogg] says he thinks that poster won the referendum, because it dominated the debate for the last few days. The establishment hated it, the posh boys at Vote Leave hated it, but it was the right thing to do. Now, I don’t think we’d have won the referendum without Mrs Merkel. But that poster reminded people what Mrs Merkel had done.”

Farage was referring to the German chancellor’s decision in 2015, at the height of the worst refugee crisis in Europe since the end of the Second World War, to open the borders to more than a million dispossessed people from the Middle East and Africa.

“After the election in 2015, [Farage’s close associate] Chris Bruni-Lowe said to me, ‘If on the Sunday before the referendum we’re discussing three million jobs, we’ve lost.’ I launched the poster, there was a bit of commentary, I had double-page spreads in five national newspapers. There was the usual criticism of it. It was only when Jo Cox got murdered that they chose to focus on [the poster] as the big issue.”

The Labour MP was shot and stabbed on 16 June 2016 in Birstall, West Yorkshire, by Thomas Mair, who shouted “Britain first!” as he attacked her. Farage told me: “I remember thinking, ‘Can I live with this?’ Not because of what I’ve done – just the hatred, not my conscience… Basically, it’s your fault she’s dead. I came in here, a bit down. It was rough, and Chris said, ‘Remember what we said last year: what’s the conversation? It’s immigration.’

“But it was obviously very unfortunate that a young woman got murdered, and all the rest of it… I don’t think her death ultimately changed the way people voted, but what it did do was kill the momentum. It did kill the momentum. Sorry, that’s the wrong word to use. It stopped the momentum. Because we had the ‘big M’ going. Momentum’s an odd thing, because when it’s going with you, you just feel it. You know it’s happening. So, yeah, that was quite a thing.”

That phrase, “quite a thing”: you could call it a euphemism.

***

Simon Heffer, a commentator, historian and authorised biographer of Enoch Powell, believes that Farage is one of the most important politicians of the entire postwar period. “Enoch was the first British Euro­sceptic,” he told me. “He kept the argument going throughout the 1970s and 1980s when most others had given up. As he faded, Farage took over, at the crucial moment when the Maastricht and subsequent treaties started to raid British sovereignty and democratic accountability. Nigel built up huge momentum over the 20 years before the referendum and, unlike the fantasists of Vote Leave with their £350m a week for the NHS, concentrated on the key intellectual argument for Brexit: the reclamation of sovereignty and the reinstitution of democratic accountability.

“And he galvanised the working class, whose criticisms of the EU and failed aspirations had been ignored by generations of Labour politicians, to support Brexit. He, not Boris Johnson or any of his crew of poseurs, was the key to the Brexiteers’ victory.”

Farage winced when I mentioned Heffer’s comment and Powell’s name. “Enoch was, er, a brilliant man,” he said, with unusual hesitancy, “but somehow the words he used, the analogy he chose, destroyed the debate [on immigration] for a quarter of a century. It made it impossible to even talk about it.”

He sensed an opportunity to reopen the debate with the enlargement of the EU in 2004, when ten new countries joined, eight of which had been part of the former communist eastern bloc. Of the existing member states in 2004, only the United Kingdom, Sweden and Ireland did not impose “transitional controls” restricting the freedom of movement of migrants from the new accession states, a fateful decision as it turned out. The New Labour government forecast that only 13,000 migrants would arrive from Poland and other eastern European countries; in the event, more than a million came to live and work in Britain as annual net migration, year after year, rose inexorably.

If – as Isaiah Berlin wrote in a celebrated essay in 1953 – the fox knows many things but the hedgehog knows one big thing, Farage is a hedgehog. The single defining thing that he knows is how to exploit people’s unease about immigration. That was his great wager: the revivifying of the immigration debate.

“The European Union and immigration wasn’t an issue before 2004,” he told me. “It was the mistake of letting in the former communist countries. Many in Ukip said, ‘No, no, don’t do that, you mustn’t do that. They’ll call us all the names under the sun.’ I knew that touching the immigration issue was going to be very difficult. But I think the impact that had on me, the family, I think all of that was bad, yeah. And frankly… the only thing that upsets me about it is that, had it been wilfully and overtly a racist message, I might have deserved some of it. But it wasn’t. It never was. It never, ever was. It was a logical argument about numbers, society.”

The emergence of Ukip destroyed the British National Party. Farage made a direct appeal to its voters. “The problem was that with the demise of the BNP, the haters on the left had to have someone to hate, and that all transferred to me.”

He doesn’t like the term “working class” but agrees with Simon Heffer that his rhetoric and plain speaking appealed to those he calls “good, ordinary, decent” people.

“The one thing I had going for me is that I’m able to cross classes. You know I do what I do, I am what I am – people like it or they don’t like it, but I’m not confined to the Shires or the inner cities. I can do a bit of both. You know how our class system is… The sort of middle, upper-middle class never say what they think to anybody, you know, just in case. But the lower down the social scale you go, the more people are very blunt in what they say and how they approach things. So, I use direct language, never trying to come across as being too clever.”

Farage respects Jeremy Corbyn because he is not a conventional career politician. “Corbyn’s a bit different, and maybe that’s why he’s working with a certain segment… Corbyn’s popularity among the young is astonishing. But he comes across as very genuine. His technique is so similar to mine in an odd way, and Trump’s. He’s very similar to Trump, the way he does it!”

What does he do?

“One, the embrace of social media. He understands it; I understand it. If you look at the social media following of UK politicians, it’s just him and me. The rest are so

far behind us, it’s almost incredible.”

Both Farage and Corbyn have more than a million Twitter followers. I suggested that Ruth Davidson, the Scottish Conservative leader, is excellent on Twitter and understands how to use social media.

“Yes, she is. Her numbers at the moment are very small, but that may change. But she understands it. The rest of them haven’t got a clue. I mean Boris Johnson! Boris should be huge on social media and he’s not. Corbyn also gets that the big public meeting works. It energises people in the most incredible way.

“And some of the stuff he said in this general election was not entirely dissimilar to some of the stuff that I said in the previous general election. On the fact that you’re living in a society where the rich and powerful are richer and more powerful than they’ve probably ever been…

“And he comes across as genuinely caring about those that are having a tough time. And that’s his big card. I’ve got a certain admiration for that. What I don’t have an admiration for is the thought of [John] McDonnell running the British economy.”

***

Before this interview, the last time I had seen Farage in person was in November 2016 at the Spectator parliamentary awards dinner in London. There, he was presented with a Lifetime Achievement Award by George Osborne. Farage had been drinking and gave a raucous, triumphalist speech during which he mocked the “pasty-faced” Osborne, whom he loathes, and then told the guests, who included the Prime Minister, Theresa May, that Donald Trump would be “the next leader of the Western world”. Farage’s comments were received with derision. “Oh, come on,” he said that night. “What’s the matter with you? That’s… the attitude you all took to Brexit. [You said] it could never happen… [But] my achievement was to take an issue that was considered to be completely wrong, perhaps even immoral, and help to turn it into a mainstream view in British politics.”

It was a fair self-assessment and he was correct about Trump winning. Farage, who understands that in the age of social media outrage cuts through, has had an astounding effect on our politics. He is blamed for coarsening and poisoning the public discourse and inflaming racism and xenophobia, charges that he rejects. Instead, he told me that angry Remainers, such as Alastair Campbell, had created what has become a foul and feculent national conversation.

“I think some of what’s happened has been appalling. Alastair Campbell, he’s almost lost reason! I mean, the classic example of what’s happened since the referendum is the death of the Polish man in Harlow [in August 2016]. So, the story is: ‘Polish man gets beaten up because of race hate caused by Brexit.’ That’s the story. It’s everywhere: BBC Two’s Newsnight even ran a report saying, ‘Nigel Farage has blood on his hands.’

“Talk about fake news… The collective shock of the liberal establishment, they still can’t get to grips with it, and they’re trying to find a reason why this illogical thing, as they see it, happened. In this country, they put it down to lies, and in America, it’s the Russians!”

Ah, the Russians – let’s hope they love their children, too, as Sting sang.

Carole Cadwalladr, an Observer feature writer who has been investigating what she considers to have been malign outside influences on the EU referendum result, is convinced that Farage is at the centre of a network of alt-right white nationalists and libertarian billionaires who are intent not only on destabilising the West but engendering hate and overturning the liberal order. “Farage has been making speeches in the US for Roy Moore, for example. Is he being paid to do that? And if so by who?” she said when we spoke.

Cadwalladr has been abused on social media by Farageists and by Arron Banks and Andy Wigmore of Leave.EU, which posted an abusive video of her on Twitter. The video has since been removed and Wigmore told me that it was meant to be a joke and he regretted the upset it had caused.

“It seems to me that Wigmore and Banks are using Trumpian rhetoric for effect,” Cadwalladr said. “It doesn’t ring true. But Farage is ideological. That’s the difference. And he’s been given a free pass in Britain for too long. It’s disturbing. It’s made me question our institutions – including the press and media. There’s no covert conspiracy with Farage. He’s part of this overt, right-wing, pro-Putin bloc. He loves Putin. He supports Hungarian demagogues.”

Farage believes that Vladimir Putin is “a strong leader”, but he would never wish to live in Russia. “You know, 120 journalists have gone missing in the last nine years… I wouldn’t want Putin as my leader, no, no, no. This is not some unqualified fan club, far from it. But, you know, he’s a strong national leader who, when it comes to playing strategic global politics, is a bloomin’ sight smarter than No 10.”

Farage spoke with enormous gusto and energy, his voice animated. “There was a piece the other day,” he continued, “that said I was the only person that connected all the dots. That I was the centre of the web. I mean, it’s just baloney… A person of interest to the FBI, etc, etc. I can tell you hand on heart, it is total and utter baloney. I have virtually no Russian links at all.”

What about Donald Trump (we met before the American president disgracefully retweeted anti-Islamic propaganda from the account of the deputy leader of the neo-fascist Britain First movement)? Farage described himself as a “supporter” and said that Trump had restored America’s reputation as a powerful nation overseas. Farage was encouraged by the administration’s programme of deregulation and tax cuts for the wealthiest and corporations, but there had been no contact between them for many months.

Farage is, however, still in touch with Steve Bannon, the executive chairman of Breitbart News and a former Trump strategist at the White House. I suggested that, with the departure of Bannon, Trumpism had lost its ideological fervour and drive – after all, Bannon has a theory of history, however demented. Farage disagreed. “Trump had that belief system anyway,” he said. “Steve may have reinforced it.”

***

Before Ukip’s post-referendum collapse into irrelevance and Jeremy Corbyn’s rise, Farage reached out to and captured a certain demographic of Labour voters, several million of whom ended up voting for Brexit. One of the most serious mistakes made by Ed Miliband as Labour leader was to underestimate Ukip, which he believed would hurt the Conservative Party more than Labour. By the time of the 2015 general election, increasing numbers of voters were abandoning Labour for the so-called people’s army.

The Labour-to-Ukip defectors were, on the whole, not city-dwelling liberals. They mostly lived in towns and did not have degrees. They were anxious about immigration, fearful of change, pessimistic about the future and weary of austerity. Caricatured as those “left behind” by globalisation, they made themselves heard at the referendum in 2016, an act of rebellion that the Blue Labour thinker Jonathan Rutherford likens to an Orwellian “tug from below”.

“Cameron would not have won the election in 2015 had it not been for the Ukip vote,” Farage told me. And if Cameron had not won the election, there would have been no referendum. “We hurt Labour far more than we hurt the Conservatives. And I remember thinking, ‘These cretins.’ The Daily Mail didn’t understand it! The Sun didn’t understand! They didn’t understand it! We were digging deep into that Labour vote. And that was the gap Miliband created. It was the gap that, Jason, you saw earlier than almost anybody, to be frank, and we did well with them. We did very well with them. So, 2015 was an odd moment, because… we had four million votes and we’d got nothing for it [Ukip ended up with one MP in 2015]. But we had a referendum!”

Call it the revenge of the fruitcakes.

On 14 November, Farage made a speech in the European Parliament during which he denounced George Soros, the 87-year-old billionaire financier who funds the Open Society Foundations, which supports civil society and liberal democracy. Soros has been traduced in his home country of Hungary and is the victim of conspiracy theories and anti-Semitism. I put it to Farage that his speech had been interpreted as an anti-Semitic “dog whistle”. For the first time, he became angry.

“Fuck off, for God’s sake. Excuse my language – but honestly, isn’t that incredible? Is this what we’ve sunk to? If you attack Soros, you’re anti-Semitic? They’re desperate aren’t they, these people? You know why? They’re losing. Because even if Brexit’s delayed, even if it’s not done properly, public opinion is hardening around the kind of things that I campaigned for, for all those years. They’re losing.”

What is it they are losing?

“Their very nice, comfortable, narrow vision of what the world is and what it should be: that’s what they’re losing. But honestly, for goodness sake. Soros? If we talk about Russian influence, let’s talk about Soros’s influence. It’s massive. David Miliband’s paid by him… And I’m being told I can’t talk about it. I mean, please. Anti-Semitic – bloody hell. Think of all the prominent Jewish people that have stood up and supported me over the course of the last few years. Sorry, that makes me angry.”

***

In his book The Shipwrecked Mind, the American academic Mark Lilla draws a distinction between the conservative and the reactionary mind. Reactionaries are, in their way, “just as radical as revolutionaries and just as destructive”. Farage is a radical and a reactionary: his instincts are destructive. He wanted to blow up the British establishment. He wanted to smash an elite consensus. He is relaxed about the idea that Britain might exit the EU without a free trade deal. He delights in describing Brexit as an “earthquake”, the aftershocks of which continue to move the ground beneath our feet.

“I’ve thought for a long time,” he told me, “that this question about Europe and our relationship with it was one that had the potential to realign British politics. In the last few months, I’ve been thinking that Brexit might not be the last earthquake. There might just be another one. There may be something seismic still to come. And it could be the Conservative Party that’s the most vulnerable to it.”

This year, far-right parties have suffered notable electoral reversals in France, Austria and the Netherlands but they have not been decisively defeated. We are not witnessing the return of a more liberal, optimistic Europe. Marine Le Pen won 34 per cent of the vote in the second round of the presidential election against Emmanuel Macron, after reverting in the final weeks of the campaign to the politics of her father, an old-style Vichy fascist. To defeat Geert Wilders’s anti-Muslim Freedom Party in the Netherlands, the centre-right Dutch prime minister, Mark Rutte, had to adopt some of his rival’s positions and borrow much of his xenophobic rhetoric. In the illiberal democracies of eastern Europe – Poland, Hungary – an ugly form of the old right has re-emerged. The Czech Republic has embraced anti-establishment populism after the ruling Social Democrats were crushed by ANO (“Yes”), an insurgent party led by a billionaire oligarch, Andrej Babiš.

Meanwhile, Angela Merkel’s centre-right government has been severely weakened in Germany and her standing diminished after the far-right Alternative für Deutschland, energised by the migrant crisis, won nearly 13 per cent of vote in the federal election in September: it now has representation for the first time and is the third-largest party in the Bundestag.

“All that has happened, especially in France, is that the rise of the far right has been paused,” said the philosopher John Gray. “What’s driving all this, I think, is that the emerging European state, or super-state, cannot discharge some of the primary functions of a state. It claims many of the prerogatives and authorities of a state, but it hasn’t got the means to deliver on the functions of a state – which do include control of borders.”

But the liberal order, although threatened, is not crumbling. The EU27 end 2017 in a stronger, more unified position than they began it, in spite of the turmoil in Spain and the intensifying Euroscepticism in Italy. In the US, the worst excesses of Donald Trump are being constrained by the courts, by Congress and the free press. And Brexit, as Farage knows, has not yet happened.

According to John Gray, “Farage is taking an Oswald Mosley-like gamble. I’m not saying he’s a fascist, but he’s reinventing himself as an alt-right politician in a culture that, despite everything, has no room for the alt-right. He has made a fundamental strategic error. Let’s say he’s arrived at a point of non-arrival. There is no alt-right position for him to connect to, because the great achievement of British politics has always been to marginalise the far right. The dark European stain that has re-emerged – and now the American stain – is altogether different. As for Farage, I think he’ll be beached in five years and probably end up in America as a shock jock.”

The man himself thinks differently. “America is very tempting. But I’m just a bit too English really! I like going to Lord’s.”

At the end of our conversation, Farage accepted that the Brexit negotiations were in trouble. He was alarmed by the economic forecasts but accepted no responsibility or blame. “It’s not Brexit that’s caused the uncertainty,” he said. “It’s Theresa May. Let’s be honest about it: the prospect of a hard-left government with McDonnell as chancellor and Corbyn as leader is scarring business.”

He believes that the Prime Minister has no conviction. “Brexit is an instruction from the electorate to turn around the ship of state by 180 degrees,” Farage said. “You cannot do that unless you believe in what you’re doing. You have to actually, passionately believe in what you’re doing. Ignore all criticism, you just have to do it. It’s like an act of going to war… And she’s managing the different wings of the party as if this is politics as normal.”

Boris Johnson – whom Farage thinks should leave politics to become an academic and television celebrity – has disappointed him. The next prime minister, he said, will come from outside the cabinet, and it could be Jacob Rees-Mogg, his choice. “Whether it’s on the question of a transition deal, whether it’s on the question of ‘go whistle’ – Boris has been very weak. The likelihood is that we will leave the European Union legally but finish wrapped up in a whole series of transition deals that mean we’re not able to take advantage of the positive sides of Brexit.”

***

The ultimate irony of Brexit is that the UK is now more at the mercy of the EU than ever. And though Farage can afford to run this ideological experiment, many of those who voted Leave cannot. He expects Labour to offer a second referendum on the unresolved Europe question in its next manifesto. “I have a feeling that Labour will fight the 2022 general election, if we go that long – we will not be fully out, we will still be in a transition of some kind – on a ticket of either, ‘We’ll have a referendum to rebalance our relationship’ (which would not be fully rejoining, but it could be a single market compromise), or an EEA compromise, or something. That’s a very realistic possibility.”

At which point, Farage may return to front-line politics, perhaps at the head of a new party or movement. “My position is this: if they really make a mess of Brexit, and if there’s a job that has to be done…”

He lowered his voice almost to a whisper and looked straight at me. “I’ve got no choice! Actually, I honestly don’t really want to. I’ve done it. I don’t want to do it again. You know climbing mountains without crampons is quite tough – you take on the establishment. But, no, if the gap is there, if it needs to be done, if the job needs to be finished, I’ll do it.”