In his spare moments on the campaign trail, John McDonnell is reading The Division Bell Mystery, a 1932 detective novel by the Labour MP Ellen Wilkinson. The mystery concerns the murder of an American financier in a House of Commons dining room. It is the sort of story one can perhaps imagine Labour’s shadow chancellor joking about in his past life as a back-bench agitator. When he is not reading reports on policy, McDonnell enjoys Wilkinson’s thriller in private, late at night on trains, between visits to marginal constituencies and his home in Hayes, the outer west London suburb. His copy was a gift from Bryn Davies, an actuary and trade unionist whom he credits as the author of Labour’s policy on state pensions (Davies, like McDonnell, worked at the Greater London Council (GLC) under Ken Livingstone). “Just to be able to read a novel again, even for half an hour, is worth it,” McDonnell, a compulsive reader and restless autodidact, told me when we met over breakfast in his constituency on 30 November, the morning after the London terror attack.

Wilkinson’s novel is an apt choice. Like McDonnell, she was a Marxist by instinct, moderated by ambition and proximity to government (she ended up as education minister in the first cabinet of Clement Attlee). McDonnell believes he is on a similar journey, which will end with him in power at the Treasury and Jeremy Corbyn as prime minister.

During his travels McDonnell has been thinking about Hugh Dalton, Attlee’s first chancellor. “I’ve just read Dalton’s autobiography. I think he’s really underrated. I know a lot of people go to Stafford Cripps [another Attlee chancellor], and Dalton was much more of a centrist… he hegemonised the debate around public ownership for a long period of time, almost for a generation.”

Back in the present, Labour is lagging behind the Conservatives in every poll – even as they tighten – and Corbyn’s personal ratings remain dire (indeed, if Dalton were still MP for the Durham pit constituency of Bishop Auckland today, then he would probably lose his seat to the Conservatives). The energy and optimism that swept Labour to unexpected gains in 2017 has this time seemed largely absent.

Many Labour MPs, some of whom have learned to trust and even like the shadow chancellor over the course of his four years on the frontbench, believe their party is on course for a defeat at least as bad as in 1983 (when McDonnell ran for parliament for the first time – and lost). Professor John Curtice, Britain’s pre-eminent psephologist, says Labour’s chances of winning a majority are so negligible as to be non-existent.

But McDonnell still believes. “We’re going to win, that’s the first thing,” he told me. “We’re going to win, we’re going to be in government. That’s the most important thing – to make sure people are aware of that, and the seriousness of our intent.”

His four decades in politics have been dedicated to the overthrow of an economic consensus that is finally unravelling under the pressures of austerity and Brexit. Boris Johnson has embraced big government and even denounced George Osborne’s austerity regime. Antonio Gramsci, I suggested to McDonnell as he waited for a bacon sandwich at a café close to his constituency office, might say he and Corbyn were winning the argument. Five years ago, they were shunned as cranks but now even the right has been forced to respond to some of their economic arguments.

“Hegemonising the debate?” McDonnell said, smiling. He half-agreed. He mentions a new “architecture of think tanks” on the radical left and a repoliticised trade union movement. “I think we have won – or are winning – the battle of ideas, and we are hegemonising the debate.”

McDonnell, who is 68, has not held executive power since the 1980s, when he served as Livingstone’s deputy at the GLC. He says now that he would have taken ministerial office under Tony Blair had he been asked. “I might have been sacked the week after, but it doesn’t matter. It would have been a yes.”

He continued: “It’s all very well and good winning the debate, but if you’re not in elected positions, if you’re not mobilising and building a huge social movement, you don’t win. You can win the argument, but you don’t necessarily win the implementation.”

Corbyn loyalists have long believed that a Brexit election would see Labour dominate the air war and win over the public with a radical manifesto, as in 2017. They are yet to be vindicated this time and privately some advisers are increasingly nervous, if not fatalistic. But on the ground, McDonnell insists, Labour “is more intense than in 2017”. He points to the campaign of Faiza Shaheen, the left-wing activist attempting to unseat Iain Duncan Smith in Chingford and Woodford Green. “Seven hundred people were up at the weekend, canvassing! In fact, it’s managing those numbers now – that’s part of the issue we’ve got.”

Labour candidates elsewhere worry more about the polls. McDonnell partly blames the media for Labour’s failure to ignite the public imagination. “They’ve had a couple more years to attack Jeremy, basically, and you’ve seen the results of that… You’re seeing intensive, personal vilification and smears. It’s not on. Of course it has some effect. It’s infected the broadcast media much more this time than it had before.” He is a supporter of the BBC but believes it has been “played” by the Conservatives.

However, he is encouraged that, as he puts it, “the polls are trending our way, which is good. We’re on striking distance in most of the polls, between six and eight points, three points either way, anything could happen. We’re on the right side. People recognise that. It’s whether or not on a cold December Thursday people will turn out.” What if Labour voters don’t? Worse still, what if they vote Conservative?

In October, McDonnell said in a GQ interview that he and Corbyn would resign if Labour lost an election. Does he agree with Unite’s Len McCluskey, who says they should not do so immediately, as Ed Miliband did after defeat in 2015?

“I’m not getting into that conversation. We’re going to win.”

One day, he hopes to retire to the Norfolk Broads and read more novels, but he does not imagine it will be soon. McDonnell has said before that he does not believe in determinism, but it sounds as if he believes Labour can win because the circumstances dictate they must. “No! Look. I’m a political artisan – that’s an expression [Bill] Clinton used, and it’s a good expression. You look at all of the work you’ve done, you look at all the different elements and factors… and I think we will confound people.” He laughs. “See you in a fortnight’s time!”

Bob Kerslake, the former head of the Home Civil Service advising McDonnell on preparing for government, appeared to suggest in a recent interview that the Labour leadership would be up for negotiation in the advent of a hung parliament. His comments alarmed Corbyn allies. But McDonnell insists Corbyn’s position would be non-negotiable and that Kerslake had been misrepresented. “Bob contacted me and said: ‘This bore no relationship to what I said.’ We’ve made it clear all the way along. We go into government. That’s what this is all about… But there are no coalitions or pacts or anything like that. If we’re in a minority position, we put the manifesto up. Let the Lib Dems or the SNP vote against a real living wage, ending austerity, investing in the NHS. Let them try it… I can’t see them voting against it. But if they did, we’d go back to the people. And they’d be annihilated.”

There are intermittent suggestions of a rift between Corbyn and McDonnell, which he rejects. “If anything, it’s pushed us further together,” he said of the four years since Corbyn’s won the leadership. “You watch each other’s backs, and we’ve been doing that for nearly 40 years. Now it’s a minute-by-minute exercise, whereas before it was maybe weekly.” In hindsight, he conceded that they could have handled the early years of leadership better but blamed their lack of control over the party.

But now the left is in charge. So what explains Corbyn’s failure to respond as quickly as McDonnell to Labour’s crises on Brexit (he has indicated he would probably campaign for Remain in a second referendum) and anti-Semitism (which he has no troubling apologising for)? McDonnell pauses, finishing his sandwich. “Different styles, that’s all. Different ways of reaching a conclusion, that’s all. And usually that means we complement one another.”

Tories are thankful for McDonnell’s unyielding loyalty. In their focus groups, he is the Labour politician Conservative voters say they like most. At the recent Tory party conference, one minister told me that McDonnell was the only Labour politician they feared. “He could beat Boris. Unlike Corbyn, he’s serious. He could run as the father of the nation – and win.”

***

Whatever the result of the election, McDonnell believes the left has “consolidated now… whatever happens in the future, the base is there to work from”. But he seems mindful of Tony Blair and Gordon Brown’s failure to nurture a second and third generation of leaders, and so is “chuffed” with the development of his protégée Rebecca Long-Bailey, Angela Rayner and Richard Burgon.

He wants Andrew Fisher, his long-time adviser who became Corbyn’s head of policy, to stay on after the election, despite having resigned his position. “I think he’s one of the sharpest minds, but more importantly than that, I think he’s one of the most dedicated socialists I’ve met. He put forward his idea that he was going to move on. I said: ‘No you’re not. Resignation not accepted.’”

Fisher will be at his side in a Labour government, McDonnell said, even if he has to be dragged into Downing Street “kicking and screaming”.

What happens if the Conservatives win?

“We could be into another lost period of time [on climate change] where we’re never able to make up the speed. Austerity will continue. It’s baked into the proposals they’ve put forward. In this constituency we have human suffering on a scale I’ve not seen for the 40 years I’ve lived here, in terms of overcrowding, homelessness, poverty.”

But Johnson sounds statist, even Gaullist, on the economy. Does McDonnell believe him? “No. Because he will do as his finance capital masters will tell him, basically, and that’s why he’s existent… whatever he says, you know he has no ideology whatsoever, and all Johnson has ever been about is the ruthless pursuit of power.”

That sounds a lot like John McDonnell: the ruthless pursuit of power.

“It’s for an ideological purpose,” he shoots back. And here, beneath the grandfatherly warmth, is the real John McDonnell. “His isn’t. His is just power for power’s sake… With us, it is about the pursuit of power – but to give power away. That’s the first thing: democratise the whole system. But secondly, for an ideological purpose: socialist transformation. But with Johnson, if you look at his whole existence, it’s the ruthless pursuit of personality. I can’t see anything else.”