Malcolm Turnbull has enjoyed some excellent polls since becoming prime minister. The man he replaced, Tony Abbott, remained consistently at low levels. So why has Turnbull proven to be so popular?

This is an even greater puzzle when it is appreciated that Turnbull was consistently trounced by Kevin Rudd when he was opposition leader in 2008 and 2009.

The puzzle of Abbott’s consistent unpopularity cries out for an explanation. We can rule out such things as the way that he walked and that he ate a raw onion. There are other factors that go back to Rudd.

Changing nature of leadership

Kevin “I’m from Queensland and I’m here to help” Rudd radiated in his public persona a sunny disposition that was reassuring and helped to convince Australians that all was going well.

Even if Rudd was a monster in private when dealing with colleagues, this did not dent his public image, which resonated with a large number of ordinary Australians.

We live in an age when many people have a desire for optimism and a positive outlook on the world. Like Dr Pangloss in Voltaire’s Candide they want to believe they live in the best of all possible worlds even as terrible events occur all around them. They do not like a world divided into winners and losers.

If we live in an increasingly Caesarist democracy dominated by the leader, then it is becoming apparent that the leader’s role is twofold – to provide safety and security for the members of the country they lead, and to do so in a positive and optimistic fashion that reassures the people nothing will interfere with their pursuit of the pleasures of an affluent society.

This is not an age that favours Churchillian rhetoric. Contemporary politicians make a big mistake if they attempt to seek opportunities to imitate Britain’s wartime prime minister.

It has now been more than two generations since Australians have faced an immediate existential threat as they did in 1942. They have experienced years of prosperity and have no greater desire than for that prosperity to continue. An appeal to “blood, toil, tears and sweat” does not resonate.

Abbott’s failure

What does resonate is a sunny optimism that all will be right, even when that appears to be far from the case. In some ways such an attitude indicates that Australia has moved away from a culture influenced by Britain to one that has much more in common with the US.

Turnbull’s unbounded faith in innovation’s power to transform Australia is reminiscent of writer and philosopher Ayn Rand’s belief in the capacity of creative individuals to bring into being a dynamic social order, a belief that inspired the denizens of Silicon Valley.

AAP/Julian Smith

The key value of this Randian creative order is self-esteem: individuals feeling good about themselves and their achievements. In such a world, the role of the Caesarist political leader is to make individuals feel good about themselves by emphasising optimism and the opportunities that lie ahead of them in the future.

It would appear that Rudd tapped into this cultural shift – this new mood – and this was the basis of his enormous popularity. By whatever means, Turnbull has also come to recognise this is the best means of becoming popular and retaining popularity.

Abbott failed to read the signs of the times. His rhetoric was Churchillian, emphasising struggle, crisis and emergency.

That there are many crises at present is undeniable. But in a 21st-century democratic country such as Australia, the general population does not want to be constantly reminded of them. They want a sunny, optimistic vision of the future in which there will be no need for blood, toil, tears and sweat. They want their self-esteem boosted.

In this regard it was a combination of Abbott’s honesty and his conservative disposition that were his undoing. Conservatives understand and appreciate the failings of human nature; they can be brutally honest in exposing those failings. This can lead to them being viewed as unnecessarily negative.

This would not matter in certain times and places. However, in 21st-century Australia – with its growing attachment to values of self-esteem and having a positive outlook – such values become problematic for a political leader who lives and dies by opinion polls.

Perhaps Abbott should have examined the career of his mentor B. A. Santamaria more closely. Santamaria was the ultimate “crisis man”. He was constantly expounding how the world was in crisis from the 1930s onwards. But ultimately Santamaria did not achieve his political goals. Australia was transformed in ways he did not desire.

This article has been amended since publication.