When John Bercow bowed out as Speaker of the House of Commons last month – in the chamber, he listened uncomplainingly for just under three hours to MPs lavishly buttering him with praise – the attention of one newspaper diarist was drawn by rumours of the sight of a somewhat unlikely observer in the gallery. Who was this attentive fellow, and what was it that he was scribbling so assiduously in his notebook? Later, a name was supplied. Apparently, it was none other than the actor Tom Hollander, who may or may not be preparing to play the former Speaker on screen.

If Bercow the biopic is shortly to become a reality, Hollander will have his work cut out. It’s not only that he’ll have to get the voice right (at his best – or worst, depending on your point of view – Bercow sounds like Charles Laughton as Captain Bligh in the 1935 movie Mutiny on the Bounty). Actually, that’s the easy part (“Order, orr-duu-rrr!”). It’s the personality that’s trickier to catch. Crikey, but he’s a strange mix. Yes, as even he admits, he is pompous, irascible, long-winded, pig-headed and, at times, utterly irritating. But he’s also quick, warm, quite funny and, most surprisingly of all for a politician, capable of great, almost dazzling, honesty. In summary, all of his emotions are perilously close to the surface and you must be prepared for spillages, even as you struggle to follow his endlessly unspooling paragraphs. In 25 years, I’ve never interviewed someone more difficult to interrupt, or more prone to tears.

When we meet in a Soho club, he has been a civilian – no longer the Speaker, and no longer the MP for Buckingham, the seat he has represented since 1997 – for three days. What is he feeling? Is it grief, or relief?

“I don’t feel grief,” he says. “But if I say relief… I don’t want to say that, because it was the greatest privilege of my life. It probably sounds trite, but it’s too early to know how I’m going to feel. I had a structure, and now that structure is gone, and it is discombobulating. In 10 years [as Speaker], I missed Prime Minister’s Questions twice, on both occasions to speak at funerals. The question arises: whither now?”

Facebook Twitter Pinterest John Bercow on his final day as Speaker on 31 October. There is talk of him being denied a place in the House of Lords. Photograph: Jessica Taylor/AFP via Getty Images

Will he go to the Lords? Convention dictates that the Speaker head to the upper house on retirement, but there is talk that pesky Bercow will be denied this privilege (talk that emanated first from the camp of an increasingly furious Theresa May earlier this year).

“That’s for other people to decide,” he says, with utmost graciousness, “but yes, I would accept, if it was offered to me. I want to write, consult, have fun, and do good. I want to ensure that I’m solvent, and I would like to speak to a wide variety of audiences around the world. That would appeal. What I loathe and fear is dancing.” So, no Strictly, then? Certainly not.

What about the election? How is he going to vote? “You can ask me,” he says. “But I’m not going to tell you.” Oh, please tell me. “No.” As I’m sure he knows – he wears the smile of an assassin – this is still an answer of a kind: given that he was a Conservative MP only five minutes ago, one would assume that if his intention was to vote Tory, he’d simply say so. But it’s also odd, this being one of only two matters on which he will hold back (the other is the allegations of bullying made against him in 2018, which he will not discuss beyond saying: “I’ve never bullied anyone in my life”). He certainly isn’t convinced the Tories will win.

“The polls are volatile. The Conservatives are somewhat ahead. But they need to be 10 points ahead to be confident of a majority.” Either way – a Tory majority, or not – he doesn’t think the election will “sort” the issue of Brexit. “This is just phase one. We’ve yet to have the debate about the trade deal between the UK and the EU, and between the UK and the rest of the world. We’re still going to be talking about this in five years, and possibly in 15 years.” Does this depress him? “I don’t easily get depressed but my personal view is that Brexit is the biggest foreign policy mistake in the postwar period.”

I don’t easily get depressed but I believe Brexit is the biggest foreign policy mistake in the postwar period

For the past three years, Bercow has presided over the efforts of first Theresa May’s government, and then that of Boris Johnson, to pass a bill on Britain’s withdrawal from the EU through the House of Commons. It wasn’t work that he ever anticipated having to do. On the night of the referendum in June 2016, watching the television in his children’s playroom at Speaker’s House, he was as surprised as anyone by the result. “A lot of us in the political class are too London-centric, and that must apply to me, too, because I thought most Brits would, in a pragmatic way, vote to remain.” Once a new prime minister had been installed in No 10, however, his amazement only grew.

“I had assumed that the architects of, and proselytisers for, Brexit would have a clear idea of what they wanted,” he says. “I was surprised there wasn’t greater clarity. I would say that Brexiteers bear some share of the blame for the delay because they did not have an agreed approach to delivery. There are, for instance, prominent Brexiteers who very publicly stated that achieving a trade deal with the EU would be absolutely straightforward.” Does he mean Liam Fox, the former secretary of state for international trade? Apparently, he does. “Quite why he thought that, you’d have to ask him. On average, a trade deal between two willing parties takes seven years to implement.”

The other problem was the attitude of Theresa May. “I don’t know if she bears me any ill will. I don’t bear her any. She was doing her best. But there was one major problem. The government pursued its approach as though it had an absolute right to parliamentary delivery – even when it was a minority government [after the 2017 election]. There is a respectable argument for saying that it was incumbent upon it to try and achieve a wider consensus; to reach out to other parties. But no serious effort was made to do that.”

Facebook Twitter Pinterest John Bercow and his wife, Sally, at the wedding of Prince William and Kate Middleton in 2011. Photograph: Graham Turner/The Guardian

Throughout all this, Bercow was determined, as he had been ever since his election as Speaker in 2009, to protect the rights of the backbencher, ensuring that the executive could not force its will on the House. Some, however, saw his actions as unforgivably political, and slanted in favour of remain. In March, to the fury of many, he prevented May from bringing a third vote on her withdrawal agreement, quoting a convention that went back to 1604. In September, he allowed rebel MPs to take control of the Commons agenda to prevent a no-deal Brexit. Unsurprisingly, he defends his impartiality in this period: “The evidence overwhelmingly shows that I was completely fair. I reject the charge that I was the agent of remainers.” However, this doesn’t mean that he’s impartial now. Far from it. He sounds to me like someone who has won a famous victory: a triumphant partisan by any other name.

Take, for instance, the prorogation of parliament last August. When he heard that this was what Boris Johnson was planning, he was in Turkey with his family. “Within an hour, I’d issued a statement to say I thought it was a constitutional outrage. That annoyed some: they said it wasn’t for me to pronounce. But if the Speaker doesn’t speak up for parliament, it’s hard to know who does.” A few days later, though, the supreme court ruled that the prorogation was, indeed, illegitimate. “An 11-nil verdict!” he says, like the Arsenal fan that he is. He looks at me steadily. “Do you understand if I say that when people know you were wrong, they find it easier to forgive you? But that when they know you were right, they don’t like it?” I nod. “Well, did they come and apologise? No, they just attacked the court instead.”

Where does he think the impulse for this prorogation came from? “I find it hard to penetrate the recesses of their fertile minds,” he says. “They say they weren’t seeking to stop debate, but it absolutely was not a normal prorogation.” His voice begins to rise. “It was declared illegitimate. Ultra vires. Not proper. And, crucially, void. I repeat: void. Void. Void. Invalid. It didn’t happen.” By now, I’m holding up a finger to indicate that, yes, I’ve got the message. But it’s no good. “Expunged from the record. Expunged. Not deleted. Expunged. That conveys an impression of some force. You can imagine a team of people with clothes doing the sponging, wiping it away.” You sound quite glad about this, I say. “Yes, I am glad about it,” he replies.

Facebook Twitter Pinterest On prorogation, John Bercow says: ‘If the Speaker doesn’t stand up for parliament, it’s difficult to know who does.’ Photograph: Stefan Rousseau/PA

There is a theory that Bercow, whose father was a cab driver, is motivated more than anything by social class. Having sometimes found himself the victim of snobbery, it is the fuel on which his engine runs, the spur for his greatest feuds and grudges. He tells me that he gets on well with Boris Johnson, whom he finds “personable”, but that he was rather less keen on David Cameron. “He was immensely capable on his feet, very dextrous, and on big occasions, he could be impressive…”

Is there a ‘but’ coming? “There is, yes, and that is that David is relentlessly tactical rather than strategic. Let’s face it, he chose to call the referendum. Was there a clamour for it? There was not. There was chuntering in his own party, but the public wasn’t demanding one. He just thought it would work for him. He has the most enormous, probably public school-instilled, self-confidence. He thinks that people like him are born to rule, that the natural order is that people like him run things, and that he is in a superior position.”

Did Cameron and George Osborne, the former chancellor, bait him? “I’m not complaining. I didn’t think George was mean. He is an icy calculator; when there is a strong difference of opinion, he’s rather clinical.” He makes his voice slightly posher, and a little bit Bond villain. “‘It’s nothing personal, but I am going to have to take you out. The exigencies of the time require it. I don’t dislike you, but the time for your political assassination has arrived. The slot has been booked, the weapon has been identified.’ David Cameron is a more emotional character. Was he mean to me? He did strongly object to my becoming Speaker. He was not keen on me at all, but then, when he stood for the leadership, I said I didn’t think Oxford-hunting-shooting-fishing-membership-of-White’s was a combination to appeal to the masses. I thought he was too privileged.”

Becoming Speaker, Bercow says, was his way of dealing with his feelings about parliamentary life. He didn’t care for the front benches [he was shadow chief secretary to the Treasury under Iain Duncan Smith, and shadow secretary of state for international development under Michael Howard], but on the back benches he felt “unrecognised, powerless… eventually I said to my wife: ‘Honey, this is a recipe for unhappiness.’” Some time after this, he had a drink with the former MP Jonathan Aitken, whose godfather, Selwyn Lloyd, was Speaker in the 70s. “‘There are other routes to fulfilment,’ Jonathan said. He almost winked at me.” Was this a lightbulb moment? “Yes.”

And did he always know he would be a reforming Speaker (among his reforms are, famously, the establishment of a nursery in what was previously the House of Commons pistol range; an increase in the use of urgent questions, whereby any MP can demand a minister comes to the dispatch box; and the abandonment of the Speaker’s traditional dress)? “No, but the reforms were obvious.” He wanted the place to be more open, welcoming and, well, normal. “I didn’t want to be hermetically sealed in fancy dress.”

Such changes were, of course, all of a piece with what Bercow calls his “political journey”, an ideological odyssey that began in Edgware, north London, where he was born in 1963. His father had a car business at the time, and his mother was a secretary, though they divorced when he was 12. He was a precocious child. One story has him sitting on the wall in front of the family home, ostentatiously reading the Times, the newspaper so huge, it all but hid him from view. He appeared on Crackerjack, the children’s TV show and, as a teenager in the audience at Question Time, he lectured Barbara Castle, accusing her of having swapped the Commons for the European Parliament because she knew the Labour party had no future. He was also a tennis player: a Middlesex under-12s champion, no less. He shakes his head, forlornly. “As Jo Durie’s coach once said to my mum: ‘I don’t think John is the most talented player on the circuit, but there are two things I admire about him: he will run for ever, and he cares massively about winning.’”

Facebook Twitter Pinterest John Bercow jokes with David Cameron at a schools tennis event on a court set up outside the Houses of Parliament. Photograph: Matthew Lloyd/PA Archive/PA Images

But he disliked school. “I was distracted by my parents’ break-up. I was at sea for a period. I’ve got some sad memories: terrible tensions between them, though much later they made up.” He only got going at Essex University, where he received a first class degree in government.

According to one commentator, by the time he was in his late teens, Bercow was “more rightwing than Marie-Antoinette”: at university, he was a hang ’em, flog ’em type, a fierce opponent of the EU, and a member of the Monday Club, a notorious fringe group that was then aligned with the Tories. He was, in other words, the polar opposite, politically speaking, of the man who, in 2002, would resign from the shadow cabinet over his support for same-sex adoption. I wonder: is he embarrassed by his former self? I expect prevarication, some talk of youthful folly. But his response is straightforward.

“Yes, very much so,” he says. “In particular… ” (It’s at this point that it happens for the first time: his eyes fill with tears and he chokes slightly on his words.) “I’m not ashamed of having argued for the privatisation of the coal industry, or advocating tax of 15p in the pound. An intellectual case can be made [for those things]. No, the only thing I feel really ashamed about is that, when I joined the Monday Club, I joined its immigration, repatriation and race relations industry subcommittee. I’m deeply ashamed of that.”

So why did he do it? “Yes, what on earth was this north London Jewish boy doing teaming up with a bunch of bigots, racists and crypto-fascists? I think I was rather influenced by my father, who in many respects was a good dad – honest, loyal, hard working, he’s been dead for 33 years last week…” Again, the eyes brim and the voice cracks. “I don’t want to rubbish my dad, but it is possible for Jewish people to be racist, just as it is possible for people of other faiths to be. Dad was a creature of his time. He had a particular view about Commonwealth immigration. One day, I went to see him, and he mentioned Enoch Powell, whom he thought much maligned. I went off and read his [Powell’s] speeches, and I thought: wow, he’s got views about markets and price controls and Northern Ireland, and I fell for that. When you are a young person, you can be attracted by an ideology.”

So what changed? How did his move leftwards begin? “In the Monday Club, I came across people who were hardcore, disgusting racists, and antisemitic, too, and I recoiled, and left. But that was in 1984. I remained strongly Conservative for a long period. I was a Thatcherite until 2001.”

In the Monday Club I came across people who were hardcore, disgusting racists, and I recoiled

It was, he says, Tony Blair’s second election victory that finally caused him to do some deep thinking. “We were smashed to smithereens. I thought: there’s something wrong, here. The electorate cared about public services that we had under-resourced, and it cared about fairness. In our attitude to women and minorities, we came across as the nasty party, and I didn’t think I was a nasty person.”

With this in mind, is he dismayed by the current mood? “Yes, there is a nastiness abroad. It is a pity.” Again, he is on the verge of tears as he gasps out the words: “In the 1950s, there were notices in pubs that said: No blacks, no Irish, no dogs. The idea that there is a regression into an… atavistic camp is a tragedy.”

Has he received the kind of threats that many other MPs, particularly Jewish members, have recently described? “Oh, yes. But it’s something I can shrug off. I don’t want to be pious. I am capable of being upset from time to time. But have I lost a wink of sleep over this? No. I lose sleep about the health of a member of my family.” At this point, beginning to sob, he asks me to switch off my tape recorder.

When he has recovered, we talk about his wife and three children, with whom he has shared the “bizarre, surreal” experience of living in Speaker’s House. Some have attributed the shift in his politics to the influence of his wife, Sally, a Labour supporter who famously has a “Bollocks to Brexit” sticker on her car. But this doesn’t seem to me quite right, not least because they met, in 1989, at a Conservative students’ training conference. Was it love at first sight? “I was very attracted to her, and I was completely amazed she had the remotest interest in me.” Naturally, his determination kicked in. “I worked at it. We had an on-off relationship, but I never gave up, and 13 years later, we got married.”

He says his wife has been a rock. But there have been – how to put this? – some challenges. She was photographed for a newspaper wearing only a sheet (power is an aphrodisiac, she said); appeared in reality TV shows; lost a libel case brought by Lord McAlpine; and had an affair with her husband’s cousin.

“Yes, we’ve had periods of turbulence,” he says. “I think it’s well known that Sally left me for a period.” How did he cope? It was so public. “In the Commons, a dear friend, Charles Walker [a Tory MP], came up to me and said: ‘I don’t know how you keep going. You must feel so embarrassed.’ I said: ‘I’m not embarrassed. I haven’t done anything wrong.’ How did I cope? I compartmentalised.” Did he know he’d get her back? “I didn’t feel a great certainty, but I thought it was worth trying. I do [love her a lot], and I’m happier together than apart.” For one last time, his eyes grow watery. “It’s easy for people to think the grass is greener. I like to stick with things.”

He must go now. The man from the speaking agency he has joined, desperate to talk to him, is tapping on the door. But I sense a certain reluctance: the result, I think, of a feeling on his part that, at last, he has the chance to show people what he is. “The way I talk is not an affectation,” he says. “It came from my father, who also spoke in paragraphs. There are certain rather stupid political scribblers, pompous, public school idiots” – I interrupt: can he possibly be referring to Quentin Letts? (Letts is the Times sketch-writer.) “Did you say Lentin Quetts?” he barks back. And now he really lets rip.

“They have this stupid, bigoted, thick-as-pig-shit idea of me as someone who is just acting a part. It’s just snobbery. Because public-school people are predestined: there’s a map for them.” Does he mean that they don’t understand that the rest of us must be more pragmatic, perhaps shaking off versions of ourselves from time to time? He nods.

“I may be odd and pompous and an irritant. But I am completely authentic.” He shows me the palms of his hands. “This is me,” he says, sounding half dazed at the thought.