Like millions of Australians, I fear I may have overestimated Malcolm Turnbull. In my case, call it biographer’s remorse. I spent the best part of a year trying to understand the man, and now…

If I was too kind, it certainly wasn’t for lack of trying. I was determined to be fair, to write the book warts and all. I called it as I saw it. Within the bounds of publishing in this country of unfree speech, I pulled no punches. His low points were covered in detail – his role in the collapse of insurer HIH, often described as his Achilles heel; his morphing of the national broadband network into a multi-technology mix, or MTM, otherwise known as Malcolm Turnbull’s Mess; his relentless undermining of Brendan Nelson.

When the book came out, while others were forecasting Turnbull would now enjoy multiple terms in office, I wrote that he would need the skills of Houdini to escape his political bind: where the public was behind him, the party wasn’t, and vice versa. Innovation was motherhood, and Turnbull was an elitist who would have trouble thinking outside his Point Piper bubble, trouble understanding the everyday concerns of most Australians. In those heady days of November–December, I may as well have been pissing in the wind. Now the gale is blowing the other way.

As Tony Windsor says: Malcolm Turnbull is like the dog that caught the car.

When I’d proposed to call the book Born to Rule, it was not tongue in cheek. After all, Turnbull was unquestionably brilliant; from an early age his parents, and later his peers, believed he was destined for the country’s top job. Writing his biography, I felt that any one of half-a-dozen chapters would mark a lifetime’s achievement for most of us. He was also, after some wobbles, a Liberal – the natural party of government.

During the federal election campaign, it occurred to me that the title still rang true, but now because it captured Turnbull’s arrogance. The sales job was beneath him. A spontaneous wit who could mix it in the pubs was now overscripted and aloof. In the televised debates, Turnbull seemed to imply members of the audience were badly mistaken. Small-target Turnbull had nothing interesting to say – the agenda was threadbare – and beyond the obvious strain of repeating himself, it seemed he hardly tried. Instead, he relied on our inevitable good judgement. We were to elect Malcolm Turnbull because he was Malcolm Turnbull. He would not persuade us. When the verdict came back, too close to call, in his graceless election night performance, Turnbull hid and sulked and even called on the police to investigate. He pointed the finger of blame everywhere but at himself. That’s arrogance. That’s born to rule. That’s Turnbull.

The haters say the Turnbull I admired was never there. His triumph over the Thatcher government, to allow publication of the Spycatcher memoir of former MI5 agent Peter Wright, was a public relations victory, they say, rather than a legal one. Turnbull’s cross-examination of civil service chief Sir Robert Armstrong, who was “economical with the truth”, was witness bullying that would never be allowed in a British court.

The Republican campaign, they go on, was let down by Malcolm Turnbull, because as its main financial supporter he insisted on calling the shots and put himself at the centre of everything despite advice that a people’s movement, rather than a merchant banker-led movement, was called for. Turnbull could not have done more for the cause, they agreed. Problem was, he needed to do less. Finally, Turnbull’s do-or-die stand as opposition leader, when he refused to “lead a party that [was] not as committed to action on climate change as I am”, was no more than a desperate grasp for an honourable way out – a chance for the captain to stand tall and go down with the sinking ship. As one party critic observed, caustically, of Turnbull’s final, electrifying press conference before he lost the leadership: “He did his pretty best.”

Turnbull’s detractors say nothing has changed and his prime ministership is an exact repeat of his failed leadership of the opposition. A honeymoon, followed by an almighty comedown. What’s more, they say, it was always going to play out this way.

I disagree. I still believe the man many of us knew and admired was there. One doesn’t need to rob Turnbull of his life’s achievements to explain what has happened to him in office. Absorbing the coverage of his first anniversary as PM, one might imagine Turnbull is a hapless loser who can never get anything right. I am not writing to defend him – the opposite – but that’s being too harsh.

We should acknowledge the degree of difficulty of being a modern-day prime minister. Nobody can tell, beforehand, who will grow in the office and who will shrink in it. We have watched a fierce opposition leader, Tony Abbott, fail as PM. We watched Julia Gillard, previously one of Labor’s best communicators, turn wooden, even ashen. We watched Kevin Rudd squib the moral challenge of our generation, climate change. Three prime ministers, each consumed by the office.

But who could have predicted the combination of gutlessness, vacuity and incompetence that this prime minister has delivered? Malcolm Turnbull was not meant to be a short-lived, bumbling, forgettable PM. He did not nurture his lifetime ambition of becoming prime minister so he could tinker with superannuation tax concessions, or pretend to lift economic growth from 2.7 to 3.3 per cent. He had the potential to be one of our great reforming PMs, from the radical centre. As Tony Windsor says: Malcolm Turnbull is like the dog that caught the car. Having sought the job all his life, he doesn’t know what to do with it, now that he’s got it.

The chronicler of Abbott’s dysfunction, Niki Savva, says Turnbull could never do enough to satisfy someone like me. He could never fix climate change overnight, bring marriage equality, close Manus and Nauru. This may be true, but it would be enough for now if his government could stop making things worse.

Speaking at a dinner in March I was asked whether Turnbull had sold out to take the leadership, and I simply answered, “Yes.” As I stepped off the stage I thought perhaps I’d been a bit harsh, too definitive. Not so. Turnbull has done his deal with the devil and will not deliver on the promise he held out to so many for so long.

What I didn’t anticipate was how spooked and cautious Turnbull had become, and how the discipline he showed in taking the leadership – keeping his powder dry in February 2015, allowing Abbott to unravel over the next six months – would manifest once in office. As much as anything he did in his career, I admired the way Turnbull swallowed his pride after losing the leadership in 2009 – put behind him the mortifying embarrassment of being duped by Godwin Grech in the OzCar affair – and rebuilt his political career. It turns out, in remaking himself, Turnbull dismantled himself. Restoring cabinet government is a sine qua non: not something to be celebrated in itself. Then what?

Turnbull’s prime ministership to date shows one thing: that intelligence is overrated. What counts, in a leader, is courage and conviction, something Malcolm Turnbull once had in spades but now seems to have lost entirely.

Where is the man who took on Australia’s most powerful businessman, Kerry Packer, spiking his cherished bid for Fairfax? Where is the man who spruiked across America the first Australian dotcom to list on the Nasdaq, OzEmail? Where is the man who spent more than $2 million and more than half a decade arguing for a republic? Where is the man who took on water reform in the Murray-Darling Basin in the middle of a terrible drought? Where is the man who defied public opinion and stood up for photographer Bill Henson in the face of a moral panic? Where is the man who stuck up for the rule of law when cabinet proposed to render foreign fighters stateless, or called for calm in the face of Daesh?

If Turnbull had called an election late last year, or early this year, he could have won seats against Bill Shorten, purely on the strength of his personal standing with the public. He would have given himself unrivalled authority within the Liberal Party, and proceeded to remake it in his own mould.

Instead we have had a ministry that lasted weeks after Mal Brough – subject of an Australian Federal Police investigation – was made minister of state. We’ve had a flubbed GST debate; a laughable proposal to up-end the federation, that lasted 48 hours; an okay-but-compromised system for optional preferential voting in the upper house; a double-dissolution election that backfired; and now the crowning ignominy, a government losing votes on the floor of the house of representatives and filibustering in the senate because it has run out of legislation to debate.

Before the election, when Turnbull told Annabel Crabb that being prime minister was the most fun he’d had in his life, I had to wonder, “Which bit?” Was it the premature surrender on every principled cause that defined him in the public mind? Was it the excitement of watching the polls dive as the public worked out they’d been had? The sense of power as police raided the home and office of Stephen Conroy and his staff, to try to bust the leakers who revealed the mixed-tech NBN was blowing out in time and budget and would need to be rebuilt? The luxury of making captain’s calls, such as killing off Kevin Rudd’s chance to be United Nations secretary-general, serving revenge up icy-cold to the man who humiliated him so badly in the OzCar affair? Or is it the simple pleasure of putting your feet under the big desk or hopping into the prime ministerial BMW which, when it turns up, is all that’s left to get around in after the Abbott government totalled Australia’s car industry?

Of course, Malcolm Turnbull might yet come good. Perhaps he can pull off a reverse John Howard, and steer a divisive gay marriage plebiscite through to a yes vote. Or give Gillian Triggs a hearing, and come up with a human rights-based solution that gets us out of the hideously expensive moral quagmire that is Manus and Nauru. Perhaps he can grasp the opportunity of bipartisanship on climate change, and turn the rubbish Direct Action policy into something fit for purpose.

But having scraped over the line at the July election, with barely a mandate, a divided party room, a slim majority and a hostile senate, it is hard to see how Turnbull can or will do anything big. Strange to say, but becoming prime minister may turn out to be the low point in his remarkable life.