Here are three things we need to know about Malcolm.

A few weeks after Brendan Nelson beat him for the dog-days job of leading a demoralised Liberal Party after its 2007 election loss, Malcolm Turnbull called Nelson’s new chief of staff Peter Hendy.

“Turnbull told me that my job was to get Brendan to resign in the next few weeks because Brendan was hopeless and he would damage the Liberal brand so much that by the time he, Turnbull, took over, the next election would no longer be winnable,” Hendy told me in 2009. “He called Nelson personally with the same message.”

Turnbull was in such a tearing hurry to fulfill his unshakeable belief that he should lead the Liberals, and the nation, that first term opposition did not daunt him, nor party room defeat, nor the normal parameters of self awareness.

He got there of course, as those who combine great ability and enormous self regard so often do, but just 14 months after he won the Liberal leadership in 2008, his party room refused to back his support for the Rudd government’s emissions trading scheme and tore him down again. He had told them: “I will not lead a party that is not as committed to effective action on climate change as I am,” and after a rebellion led by conservative climate-sceptics his party replied, “Ok then, don’t,” and installed Tony Abbott by a single vote.

Turnbull’s authority and polling popularity had already been eroded by a fateful error of judgement – possibly also rooted in his political impatience – when he relied on what turned out to be a fabricated email from the Coalition’s treasury “mole” Godwin Grech, to demand the scalps of then prime minister Kevin Rudd and treasurer Wayne Swan.

A black four months after his defeat he decided to quit politics and return to his successful business career.

Before the ink was dry on his political obituaries, he changed his mind – after advice from former prime minister John Howard. Arthur Sinodinos, now a senator and one of those who precipitated Monday’s challenge, was another who convinced him to stay, telling Turnbull “you never know your luck in a big city”.

But in order to try his luck, Turnbull had to learn that unlike his previous careers as a journalist or lawyer (representing and then leaking against media mogul Kerry Packer) or businessman (making a pile as founding director of internet service provider OzEmail) or merchant banker, a political career is better pursued by taking fewer risks and accepting that much lies outside your control. In fact some political goals can only be achieved by waiting to see if the chance presents itself at all.

Malcolm Turnbull as a 26-year-old Rhodes scholar. Photograph: Fairfax/Getty Images

“Getting poleaxed the way I did, unless you are utterly lacking in any kind of self-awareness, it does a lot of damage to your self-confidence. These are shattering blows, you can’t pretend they are not, so you have to regain your confidence, and I am confident I can make a continuing contribution to Australian public life. But I have learnt not to plan too far ahead, just to do the task at hand as best I can,” he told me in 2012.

At first he was angry, venting his rage in an excoriating blog about Abbott’s climate policy and refusing to endorse it in television interviews. Then he pushed his luck a bit with speeches full of big, attention-getting new ideas.

But for the most part the past six years have been an odyssey of self-discipline, of learning to bite his tongue and stick either to his communications portfolio or the Coalition script, whatever he thought of its contents. Six years of building bridges, and hosting endless backbench dinners, and visiting marginal electorates. A marathon effort in proving to a still deeply-wary party that he could be a team player and now a team leader.

Even to the end he was being watched like a hawk, his every move and utterance scrutinised for disloyalty or plotting or insurrection, the shock jocks attacking him and blaming him for everything (on Tuesday Ray Hadley said he was up himself because of the way he wears his shirt). The prime minister and his office viewed Turnbull and his advisers with suspicion bordering on paranoia.

But even when the backbench forced the unsuccessful leadership spill motion in February he did not blink to publicly declare his hand. As Abbott’s leadership authority crumbled and poll after poll showed Turnbull was the choice of the voters, he stuck to the script and professed full support for the leader. He had to, because to sneak one toe over the line would give his detractors cause to say “see, he’s at it again”.

As pressure grew, Turnbull supporters began taking soundings. But the challenge had to be launched quickly and with minimum collateral damage, to avoid as far as possible the cloud that followed Gillard for the duration of her leadership because of the manner in which she assumed it.

So that’s the first thing. Malcolm is the impatient, crash-through politician who learned to play the waiting game. The racing intellect who learned to keep quiet. The bull in the China shop who learned self restraint. Now that he is prime minister we will discover whether the change runs deep, whether he really learned his lesson or whether the recent curbs on his impetuousness have just been a means to an end. And he will learn whether his years of restraint have really convinced his own party to accept him.

*****

A few days after Abbott deposed him as leader, Turnbull posted a blog that began like this.

“While a shadow minister, Tony Abbott, was never afraid of speaking bluntly in a manner that was at odds with Coalition policy. So as I am a humble backbencher I am sure he won’t complain if I tell a few home truths about the farce that the Coalition’s policy, or lack of policy, on climate change has descended into.

“... any suggestion that you can dramatically cut emissions without any cost is, to use a favourite term of Mr Abbott, ‘bullshit’. Moreover he knows it.”

It was a blast of intellectual fury that a sensible policy had been scorched by people who refused to believe science in favour of no clear policy at all.

Malcolm Turnbull with his client former MI5 spy and author Peter Wright and Gough Whitlam at the launch of the book Spy Catcher in Sydney in 1988. Photograph: Patrick Riviere/Getty Images

He has spoken out in defence of climate scientists, whose work has been derided by many of his colleagues and even by Maurice Newman, the chairman of Abbott’s business advisory council, who believes the world may have entered a cooling phase.

“It is undoubtedly correct that there has been a very effective campaign against the science of climate change by those opposed to taking action to cut emissions, many because it does not suit their own financial interests, and this has played into the carbon tax debate,” Turnbull said in a speech in 2011.

“Normally, in our consideration of scientific issues, we rely on expert advice [and] agencies like CSIRO or the Australian Academy of Science, are listened to with respect. Yet on this issue there appears to be a licence to reject our best scientists both here and abroad and rely instead on much less reliable views.”

He has railed against the “dumbing down” of Australian debate in general and the debasing of smart policy for political gain.

“There is a tendency to try to dumb everything down and turn everything into a one-paragraph press release or even less, just a slogan. It’s depriving us of a substantive policy debate,’ he said in 2012, as Abbott, his then leader, was touring the country chanting “axe the tax” slogan and claiming it would add “unimaginable” costs to basic goods.

He was contemptuous of Nelson’s small target strategy even in the early years in opposition, insisting voters always needed to know what a party stood for, and that it should stand for big ideas.

Some commentators have argued that Australian politics is broken because no recent government has been able to implement a big policy change, without having it undone soon after. They say the 24-hour media cycle, that amplifies every trivial misstep and has little patience for complex argument, the Senate voting system that throws up obstreperous upper houses and the negativity of recent oppositions has just made it too tricky to do anything hard.

But actually, no recent government and no recent leader has really tried to argue the case for change before an election and then implement it afterwards and it has been a long time since voters heard a politician who could clearly explain policy in words not milled through a focus group.

On Monday, declaring his intention to challenge, Turnbull said that was exactly what he intended to do – to offer substance, not slogans, to respect the intelligence of the electorate.

Malcolm Turnbull speaking with Tony Abbott during question time. Photograph: Stefan Postles/Getty Images

So that’s the second thing we need to know about Malcolm. Everyone who wanted evidence-based policy and proper debate is about to get what they wished for. Everyone who wanted an end to the slogans better strap in for some lengthy policy prognostications. Australia’s continuing capacity to debate issues, and debate them in the centre, rather than the echo chambers of the left and the right will be tested.

But that also raises the third big question.

Turnbull has been the people’s choice to lead the Liberals because he stood out as a politician with the courage of his convictions. But it is those same convictions – in favour of a republic, same-sex marriage and a climate policy that reduces emissions in a cost-effective way – that many of his colleagues hate.

The biggest challenge of his prime ministership will be how he keeps the voters’ faith in his conviction-politician credibility, and also the faith of the party room who elected him and could depose him at any time – just like they did last time.

His most difficult balancing act will be the same policy that brought him unstuck last time. He has promised his party’s right flank that he won’t re-introduce an emissions trading scheme and will wait and see what the existing Direct Action policy achieves, interpreted by some as capitulation.

As I have reported before, the Direct Action scheme has embedded within it mechanisms that could be “dialled up” – by a government that was willing – to make it more much efficient and effective, essentially by turning it into a baseline and credit trading scheme. Turnbull’s challenge is whether he can do that over time, while answering Labor’s charge that he has sold out and without alarming his own backbench that he is bringing in his old policy by stealth.

He is social policy moderate, a republican who accused Howard of breaking the nation’s heart with the tactics that saw the 1999 referendum fail, and also a libertarian. Free speech was at the heart of the Spycatcher case, which shot him to fame as a brash young lawyer, and at issue when he took Julia Gillard to task for saying Julian Assange had broken Australian law by publishing the Wikileaks cables. (One of them recorded now Abbott-critic Mal Brough telling the then US ambassador that “Turnbull was only interested in Turnbull”)

But Turnbull is also a businessman and a true believer in free market economics. Any left-leaning voter who thinks common ground with Turnbull on climate change and gay marriage will mean they agree with most things he does is probably in for a surprise.

On same-sex marriage he has told colleagues he will stick with the plebiscite determined during the recent marathon party room meeting, but is likely to bring it forward.

That’s the third thing we have yet to find out about Malcolm. Can his party, and the electorate learn to accept the substance of his agenda. Could he possibly govern from the centre while leading a party that has veered to the right?

He had a long time to think about the mistakes he made in 2009. He has a lot to prove. His return will only be a real vindication if he succeeds.