In Tony Blair's uneven but occasionally startling autobiography, A Journey, published in 2010, there is a chapter that makes particularly interesting reading now. It covers his final, slightly besieged years as prime minister, from mid-2005 to mid-2007. "In this time," writes Blair, "I was trying to wear … a kind of psychological armour which the arrows simply bounced off, and to achieve a kind of weightlessness that allowed me, somehow, to float above the demonic rabble tearing at my limbs. There was courage in [this behaviour] and I look back at it now with pride," he concludes. "I was … not unafraid exactly, but near to being reckless about my own political safety."

The chapter's title is "Toughing It Out". Last week, during the phone-hacking trial of Rebekah Brooks, an email from the former News of the World editor emerged, sent the day after the disgraced rightwing tabloid was shut down in 2011 and six days before she was arrested. To her then boss, James Murdoch, Brooks wrote: "I had an hour on the phone to Tony Blair. He said … Keep strong … It will pass. Tough up. He is available for you, KRM [Rupert Murdoch] and me as an unofficial adviser but needs to be between us."

Rebekah Brooks jokes with Tony Blair, 2004. Photograph: Fiona Hanson/PA

As Labour leader and prime minister, one of Blair's defining characteristics was his readiness – canny or disgraceful, according to political taste – to make accommodations with powerful rightwing interest groups, not least the Murdoch press. The Brooks email, the latest in a succession of sometimes jaw-dropping revelations about Blair's behaviour since he abruptly left Westminster politics seven years ago, suggested that his ease with the left's traditional enemies had in fact deepened: into an instinctive feeling that he and they were on the same side.

With his salesman's smile and large self-belief, his ex-barrister's ability to accept and argue not necessarily compatible things, Blair has always been a slippery and restless public figure. "He's kind of a freewheeler, and always was," says the historian of the Labour party Ross McKibbin. "Being a freewheeler did him well, initially." Yet since Downing Street, Blair's "journey", already often controversial, has taken him into ever more contentious territories.

In 2008, just as bankers were beginning to be seen as the villains of the world economy, he accepted an advisory post at the American investment bank JP Morgan. According to the Financial Times in 2012, it "pays him about £2.5m a year". In 2011, through a consulting firm he swiftly created after Downing Street, Tony Blair Associates, he began advising oil-rich, authoritarian Kazakhstan. "Torture remains commonplace" there, says Amnesty International.

Last month, visiting Egypt, Blair defended the 2013 overthrow of the elected government of Mohamed Morsi: "The fact is, the Muslim Brotherhood tried to take the country away from its basic values … The army have intervened, at the will of the people, but in order to take the country to the next stage of its development, which should be democratic." Even with those last four, slightly hedged words, Blair's argument eerily echoed that notoriously made four decades ago by Augusto Pinochet and the Chilean military, when they overthrew the elected government of Salvador Allende, an event still notorious in Labour circles: "We justify our intervention to depose a government which is illegitimate, immoral and unrepresentative of the overwhelming sentiment of the nation."

The Blair government once briefly thrilled the left by allowing Pinochet to be arrested in London. But that was in 1998; the days when even the faintest whiff of socialism clung to Blair are long gone. In A Journey, published four months after the coalition took office, he wrote: "If governments don't tackle deficits, the bill is footed by taxpayers." And: "The role of government is to stabilise [the economy] and then get out of the way." A more helpful endorsement of David Cameron's state-shrinking austerity strategy from a non-Tory would be hard to imagine.

Since 2007, during straitened times for most Britons, Blair has seemed increasingly comfortable being around – and being one of – "those with money", as he refers to the super-rich in his book with telling casualness. "Blair mixes with the Buffetts and the Gateses," says John Kampfner, author of Blair's Wars, "where it is seen as matter of no great surprise that you arrive in a private jet. In Blairland, there is a sense of: 'I have become part of the Davos global elite. But I haven't been able to earn properly until now.'"

Blair hotly disputes this picture of his lifestyle. "This notion that I want to be a billionaire with a yacht; I don't! I am never going to be part of the super-rich. I have no interest in that at all," he told the Financial Times in 2012. But his intricate and often secretive set of consulting and speech-making businesses – in 2009 a Blair spokesman declined even to explain the name of one of them, Windrush Ventures, to the Guardian – have helped build a personal fortune "estimated at £70m", reported the Daily Telegraph last month. This also includes a provocative amount of property for a political figure in a crowded country currently going through one of its periodic home-ownership panics. Since moving out of Downing Street, Blair's London home has been a capacious cream and dark brick terrace in Connaught Square, near Hyde Park, with a substantial mews house behind and armed policemen perpetually guarding both. His country residence, acquired in 2008, is even grander: a Queen Anne mansion in Buckinghamshire called South Pavilion, with swimming pool and tennis court. His tycoon's tan and leanness suggest he enjoys both.

Wendi Deng: Blair has repeatedly denied rumours of an affair with the ex-wife of Rupert Murdoch. Photograph: Louis Lanzano/AP

The current issue of Vanity Fair magazine quotes an already-infamous swooning note about him reportedly written by Rupert Murdoch's ex-wife Wendi Deng: "He has such good body and he has really really good legs [and] Butt …" Rumours that Blair and Deng had an affair have been around ever since Murdoch suddenly filed for divorce last summer; Blair has repeatedly denied it and Deng told Vanity Fair she would not "engage in public allegations or respond to negative claims". But there is no denying his personal closeness both to Deng and, until the collapse of her marriage, Murdoch himself. In 2010, weeks before the general election at which Murdoch's papers did their best to drive Labour from office, Blair secretly became godfather to one of Deng and Murdoch's daughters.

"You couldn't make it up," says a former member of the New Labour inner circle. "Just when you thought Tony's behaviour couldn't get any more bizarre … His actions would be strange even for the most dyed-in-the-wool capitalist ex-prime minister, but for a Labour one, I think it looks terrible. It makes mugs of many of the people who supported him in office. He's trashed the New Labour brand."

Other Labour ex-premiers have embarrassed the party. Harold Wilson became a famously disastrous chatshow host; Ramsay MacDonald led a Tory-dominated coalition and was expelled from the party – Blair has not done anything so traitorous, so far. Yet McKibbin says that all of them "had a different attitude to money. Wilson was pretty poor when he died. [Jim] Callaghan had quite a nice farm, which he retired to." Even the derided Gordon Brown's near-silence since losing office looks steadily more dignified with each controversy about Blair's new career.

In fact, it does have some high-minded elements. His website lists the Tony Blair Faith Foundation ("to promote respect and understanding about the world's major religions"); the Tony Blair Sports Foundation ("to increase participation in sport … particularly by those who are currently socially excluded"); work on "African governance" and "breaking the climate deadlock"; and his role as representative of the international quartet, on behalf of the UN, EU, the US and Russia, to try to find a peaceful settlement between Israel and the Palestinians.

Blair is not paid for any of these roles, which generally receive less press attention. He argues that his richly rewarded commercial work is undertaken mainly to subsidise them. And he says he takes great care to avoid conflicts of interest: for example, doing no business in Israel or the Palestinian territories, to avoid damaging his credibility there as the quartet's representative.

The problem is, his credibility as a sort of freelance super-diplomat in the Middle East and elsewhere is damaged already. His almost unqualified support for Israel as prime minister, his crucial backing then for the invasion of Iraq, his fundamental agreement with the bellicose foreign policy of George Bush – all this historical baggage follows Blair around. "It would be hard for him to move into working for more liberal international institutions," says a former ally, "because he's toxic."

Tony Blair's £4m country home: South Pavilion at Wotton House in Buckinghamshire. Photograph: John O'Reilly/Rex Features

Nor does Blair show much sign of having thought afresh about the shape of the world since leaving office. Last summer, during the clamour for Britain to intervene militarily in Syria, he was one of the loudest hawks. Ed Miliband ignored him. In much of its foreign and domestic policy, Labour is moving politely but firmly away from Blairism now. Miliband's populist leftwing attacks on capitalist "predators" contrast with Blair's insistence in A Journey that during the financial crisis "the 'market' did not fail". Later in the book he adds: "The danger for Labour [after losing the 2010 election] is that we … move decisively … to the left. If we do, we will lose even bigger next time."

We will see. But for now the opinion polls suggest that Blair's warning may look foolish when the votes are counted in 2015. Either way, many in Labour have stopped listening to the man who led them to three handsome general election victories, and who was once one of the most popular figures in British political history. "People I know in the party don't think about him very much nowadays," McKibbin says.

Blair is only 60. One of his problems is probably that he left Downing Street too young. Callaghan was 67 when he stopped being prime minister in 1979. But British political leaders, like bosses in many fields, have become steadily younger since then. Just like Blair, David Cameron and possibly Miliband too will have decades to fill after the Downing Street removal van comes.

A well-connected New Labour source says: "Someone who knows Tony very well said to a friend of mine recently: 'He's very unhappy.' It's a false life he's leading. And the rich are boring. What has happened to Tony has elements of tragedy."

Other Blair-watchers see his trajectory differently. "I don't think what people think of him has ever worried him too much," says McKibbin. Blair's Connaught Square house is right next to Edgware Road, one of the centres of Arab London, and of potential outrage, at the very least, at his Middle East stances. Meanwhile the plush London offices of all his overlapping enterprises are right across from the US embassy in Grosvenor Square, as if to taunt critics who claim he is America's poodle.

Last month, Blair was eating with his family and some friends in a London restaurant when a barman working there, inspired by the website arrestblair.org, tried to perform a citizen's arrest on him, for "a crime against peace … namely your decision to launch an unprovoked war against Iraq". The Daily Mail reported: "Blair attempted to engage in a debate before one of his sons went to get security."

In private, ex-prime minister Blair may be tormented and unfulfilled, but in public he remains a smooth performer. Will that be enough?