Shortly after David Cameron’s election as Conservative leader, we were throwing around ideas to underscore his determination to change the party. None seemed to excite much enthusiasm. “Have you not got anything that will annoy the right a bit more?” he asked. Such was the mood in his early days, driven by his fierce desire to plant the Tory flag firmly on the centre ground and stop “banging on” about Europe.

Eleven years later, this dream lies in tatters. Cameron has become the third Tory prime minister in succession to have their term in office torn apart by bickering over Brussels. He leaves Downing Street at least two years earlier than planned, unable to solidify his legacy as a one-nation reformer and with achievements overshadowed by that needless referendum. He sought to appease the hard right on Europe; instead, they ripped open this running sore to force him from office.

Theresa May: unpredictable, moralistic, and heading to No 10 Read more

As my former boss departs after six years as prime minister, we can at least give thanks that a rightwing coup against him ultimately failed. Yes, a cabal of social conservatives ousted Cameron, aided by foolish moderates who let personal ambition override common sense. Now our country must suffer the corrosive consequences of Brexit. But at least this faction has failed to reclaim the party and put one of their number in power.

The idea of Andrea Leadsom in Downing Street, with such anachronistic views on issues from feminism and gay equality to workplace regulation, was insulting to the electorate. She was both inadequate for the job and grossly inexperienced – and has been left humiliated by her imploded campaign. Yet 84 Tory MPs proposed her to the wider party as the person to heal the wounds of the divisive Brexit debate and take the country forward into a new relationship with the world. This reveals the shallowness of attempts at modernisation.

The rapid demolition of Leadsom’s lamentable campaign shows the Tories remain hungry for power. It offers a telling contrast with the tragic contortions of Labour, which seems determined to spin out its leadership crisis for as long as possible. Now we face the intriguing prospect of the enigmatic Theresa May as prime minister, her steel shown by six years at the Home Office and a smart strategy plotting her dogged path to Downing Street.

It is rare a politician spends so long in the public eye yet remains so little known. May is seen as a tough operator, unafraid to tell Tory delegates they were viewed as the nasty party, and to deliver harsh truths to the police. I fear her approach to immigration, given a poor track record that includes the vile “Go Home” posters, a reluctance to compromise even on foreign students, and a perceived need to appease voters stirred into fury against migrants and refugees. Unfortunately politicians on all sides prefer to blame foreigners rather than themselves for long-term failures on public services.

'Nasty party' warning to Tories Read more

Yet she may surprise people. As a vicar’s daughter with the air of a sensible headteacher who uses minimal small talk, May is often called a traditional shire Tory. Certainly she is pragmatic, weighing up decisions carefully and sticking to them – and woe betide anyone getting in the way of her tight-knit team. She is a control freak, who must learn the art of delegation in Downing Street. But although not a driven ideologue determined to shake up the country, she is more complex and interesting than the usual caricatures, with their whiff of patronising sexism.

Do not forget that it was May who 14 years ago bravely delivered to a Tory conference that controversial message that the party was seen as out of step with changing social dynamics. As campaigners will confirm, May is energised by unlikely issues such as police brutality in custody and the racial bias of stop-and-search. And she was singled out for praise by Liberal Democrat ministers for her crucial support of gay marriage, and by Labour’s Andy Burnham for fighting the Hillsborough cover-up.

As a wily politician, May knows that it makes sense for Conservatives to be centrists rather than frothing away angrily on the right fringes, especially when Labour seems focused only on its own problems. Note how in her speech today, before Leadsom pulled out, May said that her government would deliver for those born poor, or who are black, or are white working-class, or women, or suffer mental health problems.

She believes a sustained attack on corporate vested interests can help reunite the country and reconnect with voters disenchanted by “elites”, especially at a time when even millionaire bosses of multinationals believe their vast pay has become immoral. This also offends her own sense of decorum.

But one issue will hang heavy over her premiership: how to resolve the nightmare of Brexit. An aide suggested the inevitable recession will depress immigration, allowing her to tilt the balance towards open markets rather than closed borders while reassuring the electorate. Yet with a tiny majority and an embittered right, she faces a big challenge to find solutions that balance political reality with economic need.

Sadly, for all the fine talk May is likely to be another Tory prime minister who finds their tenure wrecked by little Englanders playing selfish games over Europe.