A few weeks ago, a backbench Tory MP, convinced that Theresa May’s time was up, had a meeting with Boris Johnson. The MP was honest with the former foreign secretary during their 30-minute chat. “I said to him: ‘I’m not going to commit to anyone yet. I have rubbished you and trashed you in emails to constituents. I’ve been quite rude about you in interviews about the way you behaved in Brexit votes.’ I said to him: ‘One problem is you’re a receptacle for every wack job and loon in the Tory party.’ He took it quite well.”

The meeting was revealing, not so much for what Johnson said in an attempt to win over the MP, but for the fact that he had found time to have a lengthy meeting with a sceptical colleague. Despite it all, the MP conceded to Johnson that he now faced a dilemma: “I find myself sitting here thinking I might very well vote for you.”

Johnson’s previous tilt at the top job in 2016, brought to a disastrous end by Michael Gove’s decision to run against him, was marked by chaos and calamity. Even the decision to pull out, he now concedes to allies, was an error. Johnson is understood to have apologised since to supporters who begged him not to throw in the towel.

His time as foreign secretary was punctuated by gaffes and saw him largely written off as a future prime minister. Yet Johnson has not only managed to re-emerge as the frontrunner to replace May: he has done what some thought impossible, conducting a disciplined, strategic and (so far) relatively error-free campaign that has seen him build more support among MPs than any other candidate.

Facebook Twitter Pinterest In a still from his recent campaign video, Johnson is seen with a woman who admits she is not a Tory voter...

So what has changed? In interviews with the Observer, several people who have been close to Johnson since 2016 and MPs who have observed his latest bid for No 10 described a campaign that had been long in the planning, with a larger and more cohesive team and a candidate willing to follow orders and put in the hours. Several said there was a sense from Johnson that he knew he was fortunate to have another chance. “Boris is a figure who defies political rules – and the fact he has another chance is just another example of that,” said one.

Johnson still has key lieutenants from his 2016 leadership bid, notably MPs Jake Berry and Nigel Adams, who remain close at hand. He also speaks to Lynton Crosby, the political strategist who masterminded his London mayoral wins, every day – but as a friend, rather than formal adviser.

Yet it is the new arrivals since 2016 that appear to have helped steady the ship. In a race that is all about winning support from Tory MPs, parliamentary expertise has been drafted in. James Wharton, a former MP popular among colleagues, was recommended as a figure who could boost Johnson’s once dismal reputation among his fellow Tories. Gavin Williamson’s sacking as defence secretary by May also meant there was a former chief whip on the market. Johnson’s special adviser Lee Cain has been charged with the unenviable task of limiting his boss’s newsworthy outbursts and has recently been given reinforcements.

The next element was understanding how Johnson could go about changing the minds of enough colleagues to secure a place in the run-off between two contenders, who will face a vote of Conservative members in July. Here, they had a stroke of luck. After finding himself on the backbenches under May, Grant Shapps, the former party chairman, found the time to read the multi-volume biography of former US president Lyndon Johnson by Robert Caro. Apart from gaining a detailed knowledge of Texan hill farming, it gave Shapps an interest in Johnson’s first rule of politics – you have to be able to count. From the backbenches, Shapps decided to test his skills – building a spreadsheet of MPs and attempting to predict how each would vote in the no-confidence motions and meaningful Brexit votes. His results were close to the mark.

He then used the same system to map the leadership election, looking at how the electorate of 313 Tory MPs would break down. It led to the creation of “the spreadsheet” – a detailed account of where MPs stand and how winnable they are. Persuaded that the Johnson campaign was serious, Shapps joined the crew and his data has been used to put MPs in different groups, with “pivot tables” used to show what kind of campaigning each should be subjected to – and which are a lost cause.

It has helped the team direct their efforts – and seen Johnson complete as many as 16 appointments with colleagues a day. Breakfasts. Coffees. Lunches. On Monday night, his team had even moved on to assembling a dinner that included some MPs who were already publicly backing another candidate. Few have been safe from the Johnson charm offensive.

Facebook Twitter Pinterest ...whom he canvasses for support...

One MP who backed Johnson early on said: “I said to Boris, ‘I really like you. I think we do need a change in direction, but frankly I’m not attaching myself to a campaign that was as chaotic as the one in 2016 or frankly, since then. It’s just out of the question.’ I took a gamble on him being true to his word and he has been. He’s been 100% focused.”

Many figures were reluctant to speak on the record – a sign of the discipline at play and the importance attached to “speaking with one voice”. However, Simon Clarke, a pro-Brexit MP and an early supporter, agreed to give his account of the Johnson revival. He said he was approached by Johnson at the end of last year. “He is someone who doesn’t treat you as if he is the big cheese and we’re the bloody infantry,” he said. “It was very much a two-way conversation. He was interested in marginal seats in the north that voted Leave – and how to keep those seats and win others like it.

“I think he’s been tempered by the experience of 2016. He is at ease that this is a challenge and that the leadership is a hard fight. The bottom line is we’ve done our homework. We’ve spoken to people, but we’ve genuinely tried to listen as well. We’re not doing the hard sell.”

Luck has undoubtedly played a significant part in his campaign. The most obvious factor in his success is the arrival of Nigel Farage’s Brexit party, which has helped to frame the central question in this contest – namely, who is best equipped to see off that threat from that party. “The main difference, I suspect, is that the situation is so stark,” said Clarke. “People recognise that this is an existential challenge for the Conservative party. All the available polling evidence suggests there is one person with the policies and personal skills to connect to the voters that we have temporarily been sundered from. It’s not a difficult sell to colleagues. People recognise that things have not worked.”

Facebook Twitter Pinterest ...and comes up trumps.

Then there are the candidates who take a harder line on Brexit – most notably Dominic Raab, whose presence has allowed Johnson’s team to paint their man as the person more likely to reach out to Remain voters should Brexit be secured. One figure from the One Nation caucus of moderate Tory MPs explained: “We have to be asking ourselves, what happens in a Boris vs Raab contest? In that scenario, isn’t it Boris who is the more likely one to compromise? To at least delay no deal so we can manage it? That’s the dilemma.”

One pro-Remain MP backing Johnson, who is well aware of all the risks involved in supporting him, was nervous last week as his candidate appeared at a private hustings. “I thought, am I about to look like a complete idiot for supporting this guy? But actually, what we saw was, yes, a bit of humour. But there was also some gravitas there. It was a relief.”

Many obstacles lie ahead. With Gove’s campaign disrupted by his admission that he had taken cocaine, Johnson’s own drug use will again come under scrutiny. On the BBC show Have I Got News For You, he said he had been offered cocaine at university, but added a typically Johnsonian get-out clause: “I sneezed so it didn’t go up my nose. In fact, it may have been icing sugar.” Asked later in a GQ interview to clarify whether any had gone up his nose, he replied: “It must have done, yes, but it didn’t do much for me I can tell you.” He has admitted smoking cannabis. His team will be hoping that this is just another issue from Johnson’s back catalogue of indiscretions that seem to have no bearing on his bulletproof appeal to the membership.

But then there is the elephant in the room of this leadership election – the Johnson Brexit policy. He has vowed to Leave, with or without a deal, at the end of October. Yet even MPs backing him already accept that parliament will use all means available to stop him. The official line from the campaign bunker is that Johnson will reveal all in due course. Supporters are adamant he will not delay Brexit again. “It’s not for me to answer [how he will ensure Brexit happens by the end of October], but I would say that his commitment to that date is genuine,” said Clarke. “The message is the same in private as it is in public. I think in a way, the onus will pass to parliament.”

One cabinet minister pointed out that not only would parliament stop a no-deal in October, but that the changing of the European commission over the summer would mean Johnson would have no one to agree a new deal with. “There will be no one in Brussels,” they said. “Would [current lead negotiator] Michel Barnier really have the mandate to agree a new deal, even if he wanted to? It is unlikely.”

The big problem for Johnson’s campaign is that, in all probability, he is currently engaged in the easy bit – winning over Tory MPs and members in circumstances tailor-made for him. However, Opinium polling for the Observer shows the huge doubts harboured by voters as a whole. The pollster tried to measure how voters regarded him by compiling a “net agreement” figure on a series of personal qualities – the difference between those who agree he has the quality and those who do not. Voters saw him as determined, returning a net agreement of +28%. That is where the good news ended. He had a net “likable” rating of 0%, -7% for “says what he means”, -13% for competent, -18% for “trusted to take big decisions”, -26% for “truthful” and -30% for “stands up for the most vulnerable”.

And Johnson’s volatility as a candidate means that some cabinet ministers are advising his opponents to make sure there is a full calendar of members’ hustings throughout July, in the hope that his discipline will fray. “There must be a chance that he will begin to be challenged,” said one. “They must go for a full contest.”

BORIS’S BLUNDERS

September 2012

Apologises for a 2004 Spectator article when he was editor that claimed drunken fans were partly to blame for the Hillsborough tragedy.

April 2016

Suggests the “part-Kenyan” US president Barack Obama had an “ancestral dislike” of Britain.

May, June 2016

Says Britain sends £350m a week to the EU – a claim contradicted by the UK Statistics Authority.

November 2017

Apologises for saying Nazanin Zaghari-Ratcliffe, a British-Iranian detained in Iran, had been teaching journalism. The then foreign secretary’s claim contradicted her family, who were adamant she had been on holiday.

August 2018

Accused of Islamophobia after saying Muslim women wearing burkas “look like letterboxes”.