Boris Johnson aches to be prime minister. Every move he makes and every sentence he writes is viewed through the prism of his unyielding ambition. If news of his tempestuous private life erupts, it is seen as a move to clear the decks for a leadership bid. If he writes provocatively on Brexit, his words are interpreted similarly. He so obviously yearns for the top job in British politics.

On the basis of what has happened in the past, that means he will never get it. I have been looking at political figures who were seen at some point in their careers as future prime ministers for a BBC series. Many of these prime ministers we never had were more charismatic by miles than some of those who did get the job. A few would have been outstanding, but they never got to No 10.

When an embryonic challenge seems overwrought and frenzied, the subject of the hysteria never wins

One of the reasons is that, for some, their vaulting ambition was too transparent. They could not escape being seen through the lens of leadership-scheming, to the point where they ceased to have weighty authority. Their actions were viewed solely as self-serving, sometimes unfairly. After he walked out of the cabinet in January 1986, there was little subsequent analysis of Michael Heseltine as a policymaker or what he stood for. The only interest in him was as a potential successor to Margaret Thatcher. For a brief period in the mid-1990s, Michael Portillo could not stand up without the action being interpreted as a leadership bid. He wanted it too obviously, even allowing his allies to set up a phone bank for a possible leadership challenge that he never made. He appeared to be interested in his own ambition alone, to the point where he ceased subsequently to be ambitious. At different points, the actions of Roy Jenkins, Denis Healey and Kenneth Clarke as chancellors were seen as leadership plots. The prism undermined them as weighty chancellors.

None of the prime ministers we never had displayed such transparent ambition as Johnson, dominating media coverage while leading loudly and vaguely the Brexit charge against Theresa May. It is all too crudely obvious. Overt egotism annoys colleagues, who have the power to stop the hunger being sated. In contrast, those who become prime minister often do so without a frenzied build-up. Margaret Thatcher rose speedily and unexpectedly. Harold Wilson and Tony Blair acquired crowns after the sudden death of leaders who were expected to be in place for many years. The relatively self-effacing John Major was only settling down as chancellor when Thatcher fell with a Shakespearean jolt that none had anticipated a few weeks earlier.

The prime ministers we never had were stopped for other reasons too. A lot of them became prominent for representing one wing of their party. Their noisy identification with a cause made them impossibly divisive choices as leader. Clarke was too committed to one side of the Europe argument. Healey was admired by one half of his party and loathed by the other. Conversely, Tony Benn was loved by those who loathed Healey and loathed by those who loved Healey. Johnson is more divisive than any of them, being publicly and viciously condemned by some colleagues.

Such dissenters do not become prime minister, partly because perceived disloyalty is rarely rewarded. In the mid-1970s the then prime minister, Jim Callaghan, told Benn that he had all the qualities to be the next Labour prime minister, but the party would not forgive him for continued disloyalty to the leadership. Sure enough, when Benn stood for deputy leader, in 1981, he lost the support of key MPs including Neil Kinnock. Heseltine was viewed by some party members and MPs as disloyal to Thatcher. He was a model of restraint compared with Johnson.

Nor is Johnson acting authentically. Authenticity is a key to becoming prime minister, with all the scrutiny that accompanies those who seek the role. We know what Johnson really thinks about Brexit because he told us. His first article declaring his support for the Brexit campaign in the 2016 referendum was stuffed full of qualifications. It is worth reading in the light of his new, evasively hardline approach. From a different part of the political spectrum, Ed Miliband was seen as the favourite to be the next prime minister in the 2015 election, but he recognises now that he struggled to be authentic. He wishes he had been truer to his radical self. Johnson acts as if he is Donald Trump, when he was, at least in relation to Brexit, more intelligently torn than Trump could ever be about any policy area. He will be seen through.

There are many other factors that explain why some do not become prime minister. Challengers rarely seize the crown. Timing and context matter hugely. Above all, when an embryonic challenge seems overwrought and frenzied, the subject of the hysteria never wins. Perhaps in the era of Trump and Brexit, everything has changed and none of these factors applies any more, but I suspect a lot of them still do. If they are relevant in this wildly unpredictable age, I expect to add Johnson to my list of Prime Ministers We Never Had. More than any other figure with that unsought-for title, he has all the qualifications.

Steve Richards gives a series of unscripted talks to camera in Reflections: Prime Ministers We Never Had, on BBC Parliament and BBC iPlayer from 17 September. The series begins with Michael Heseltine

• Steve Richards is a political columnist and broadcaster. He regularly presents Radio 4’s Week in Westminster