It can safely be argued that the past three Australian prime ministers, Tony Abbott, Julia Gillard and Kevin Rudd, have not been great leaders. And that the trio who preceded them, John Howard, Paul Keating and Bob Hawke, were among our best.

Australia’s history is almost long enough now to spot its repetitions. Tony, Julia and Kevin are the Billy McMahon, John Gorton and Harold Holt to John, Paul and Bob’s Robert Menzies, Ben Chifley and John Curtin. Three prime miniatures, each fated to short, disappointing stints in office after an extended incumbent cycle that changed the nation.

The first thing to note about our present malaise is that you wouldn’t wish a world war or economic stagnation on Australia to cure it. Keep your fingers crossed that the electorate is imposing the equivalent correction by continually turning over bad governments until the system reconnects with the people.

The second thing to note is that Abbott, Gillard and Rudd cannot offer the excuse of a weak economy for their performance. Like McMahon, Gorton and Holt in the late 1960s and early ’70s, they had the advantage of governing in growth. The deep recessions occurred on the watch of the lions. Menzies survived two – the bust following the Korean War wool boom in the early 1950s, and the policy own goal of the credit squeeze in 1961. Hawke was re-elected in 1990 despite the blunder of the 17% home mortgage rate. Keating took unemployment to 11% yet was still able to increase Labor’s majority in 1993.

The governments of Curtin, Chifley and Menzies and Hawke, Keating and Howard had, in common, an agreed economic model. The former was the state-based reconstruction of Australia after World War Two, the latter the market-based revival after the multiple crises of the 1970s and early ’80s. Both eras commenced with the Halley’s comet of federal politics – a long-term Labor government – and ended with the complacency of a conservative government that secured one too many re-elections. Interestingly, the time spans are identical at almost 25 years apiece.

Each of these prime ministers understood the acquisition, retention and use of power. But they had political apprenticeships that are inconceivable today. Curtin was Opposition leader for six years before becoming prime minister in 1941. Menzies was prime minister before he was ready, and spent eight years in the wilderness before he earnt his second chance in 1949. Hawke, Keating and Howard had two decades each in the public eye before they ruled.

Rudd was prime minister after less than ten years in parliament; Gillard after 12 years. Abbott is the exception in this group. He had been in parliament for two decades before declaring that the grown-ups were back in charge, although most of his internship was spent attacking Labor.

This is where the past offers no comfort. The end of the Curtin–Chifley–Menzies era still carried the promise of national renewal because the Labor Opposition still felt an obligation to offer an alternate program. Even after Gough Whitlam was dismissed in 1975, there was continuity of reform. Malcolm Fraser delivered the last rites to the White Australia policy when he accepted the Vietnamese refugees ahead of public opinion. Does anyone believe Gillard and Abbott will forge a post–prime ministerial friendship like the late Whitlam and Fraser, and lecture the next generation of politicians about values?

Rudd did offer a fresh start when he brought Labor back to office in 2007. But his program was dismantled by his own hand. He ducked the “great moral challenge” of climate change and “lurched to the right” on asylum-seeker policy after assuring the community he would do no such thing. He regrets both mistakes now.

These were not routine political backflips but rather the sacrificing of what were presumably lifelong ideals. Imagine Keating increasing tariffs to save the 1993 election, or Howard giving One Nation voters back their guns in 1998.

Holt’s death and the failure of the Gorton experiment let the Liberals promote McMahon in 1971, which is now political shorthand for a government that’s out of ideas. Rudd’s double implosion spread the McMahon curse across both parties, delivering government to leaders who should not have been in the queue, at least not in 2010. The contest between Gillard and Abbott diminished us, and the system is still processing the shock.

Gillard did not feel the need to explain herself, either to the caucus beforehand or to the public after she took Rudd’s job. She was the first Labor prime minister without an identifiable program. Converting to the cause of climate change after the election proved to be too smart by half. She regrets those mistakes now.

Her leadership model was the antithesis of the Labor ethos: Gillard claimed power first, and thought about what to do with it afterwards. But this might be the new Labor way. Bill Shorten has been conspicuous by his politicking. Inspired by Abbott’s example in Opposition, he seems determined to whinge Labor back into office.

Abbott was an accidental leader in 2009; the Liberal party room preferred Joe Hockey at the time but he was counted out first in the three-way ballot with Malcolm Turnbull. Hockey had wanted to give his colleagues a conscience vote on the emissions trading scheme, which would have seen it passed into legislation. But they settled for the unity of negativity.

Then Rudd fell, and Abbott was suddenly an alternative prime minister. One of his dearest supporters in the media has told me that Abbott was not ready to govern in 2010, and was fortunate to have an extra three years to prepare for office. You just shake your head at a two-party system, because the 2013 campaign between Rudd and Abbott was as trivial as its predecessor. And Abbott was no closer to being match-fit for government.

Abbott was the first Liberal prime minister to run against the very institution of parliament. Fraser went close in the constitutional crisis of 1975, but he could always say he was protecting the institution from a reckless regime. Abbott made Australia ungovernable as a precondition for taking power. His colleagues were too smitten with the shortcut he offered them back to office to ask the first, obvious question: what would stop Labor and the minor parties from replying in kind? Especially when the Coalition would have to break promises if it wanted to balance the budget.

These are not quirks of individual personalities, or the collective fault of the media. They are elementary errors of governance. The problem is in the parties themselves.

Under Rudd, Gillard and Abbott, Labor and Liberal swapped identities. Labor became the cynical party of the focus group, Liberal the edgy party of zealotry. These shifts had been coming for some time. Labor people have worried about the machine taking over the caucus since the late 1990s. Liberal people will admit, very quietly, that the ideologues have held the majority in the party room since the middle of the last decade.

If the transplant had been a clean one, the system might be working today, pitting genuine conservatives against inspired reformers. In that debate, national problems could still be solved.

But each side has taken the most contemptible part of the other’s persona. The Liberal Party is the Labor of the 1950s, moralising on issues that are of zero interest to the community. Labor is a digital version of the Andrew Peacock Liberals, all tweets and no meat. The prospect of an early return to power prevents whichever side is in Opposition from doing the necessary internal work of purge and reform.

The Hawke–Keating–Howard consensus of the 1980s and ’90s is misunderstood by many politicians today because it is seen with the hindsight of an economy that worked. It is assumed that there must have been some sort of détente between Labor and Liberal to wave through reforms in the national interest. That is not what happened. The main parties didn’t applaud each other’s agendas. What they agreed on was the nature of the challenge. Then they argued over the detail.

Today if one side says the nation faces a crisis, the other feels compelled to deny it. Think the Coalition on climate change and Labor on the structural hole in the budget. Everything is contested, not as a means to a policy end, but simply to annoy the other side. This politics is American in its effect – tribal and petty, fanatical and consciously dumbed down. The Australian voter has picked it for what it is, even if the two parties don’t realise it yet.

The best of Australia’s political culture was pragmatic in the true sense of the word. It resisted the extremes of Reaganism and Thatcherism in the 1980s, and avoided the celebrity traps of Clinton and Blair in the ’90s and noughties. Hawke and Keating could be trusted to free up the economy while maintaining a decent social safety net. Howard kept us relatively sane with his ordinariness. When he made mistakes – for instance, on petrol prices and the GST – he said sorry. Not always. His contrition was selective, and it could leave a hole in the budget. But he said sorry often enough to maintain public faith in our system.

Today neither side wants to concede even the smallest human error for fear of losing that minute’s news cycle. They are so immersed in the game of politics that they have imported the very thing the Americans hate about their own system: its partisanship. And then they wonder why the electorate has been so volatile.

Yet the flipping of leaders, and governments, risks making the system even more American. Both sides know their bases are shrinking. Logic would suggest that they would try to repair those bases, or build on them with new supporters. However, the two-party system is a mandated duopoly. Compulsory preferential voting means most seats still boil down to a choice between Labor and the Coalition, even if the primary votes of each have a three in front of them. That creates a perverse incentive for the parties to play to their diminished bases. This, in turn, opens a gap in the political marketplace for splinter parties. And so the two-party system becomes more polarised, as Labor competes with the Greens and the Coalition competes with the parties to its right.

One of the reasons the two-party system worked in the reform age of the 1980s and ’90s was that both sides chased the mythical swinging voter in the centre. It was conceit, of course. But it did ground our politics. The danger now is that both sides no longer care enough about the voter in the centre to know how to speak for the nation.

Rudd, Gillard and Abbott may not be the deviation from the mean. They may be the new normal – disappointing leaders with limited shelf lives.