When George Osborne interviewed Margaret Thatcher’s biographer Charles Moore on 15 October, he asked of the poll tax: “What’s your advice to a chancellor who finds that there’s a terrible policy being implemented by the government? Do you try to make it work or do you kill it off?” Osborne’s quip was greeted with much laughter, some of it nervous.

The Chancellor’s joke betrayed his confidence. His Conservative party conference speech had been well received, confirming his status as the early front-runner to succeed David Cameron as prime minister. The House of Commons had voted in favour of the tax credit cuts by a majority of 35, with just two Conservative MPs rebelling. Frank Field, the Labour chair of the work and pensions select committee, presciently warned that the issue would “catch fire”. But to every objection, the Chancellor and his allies replied that there would be no concessions.

Eleven days after his conversation with Moore, Osborne found himself desperately trying to make a “terrible” policy “work”. Less than an hour after the House of Lords defeated the government over tax credits, he announced that there would, after all, be help “in the transition” to the new system. The peers’ rebellion was the proximate cause of his volte-face. Yet it merely hastened an outcome that was regarded as inevitable.

The Conservatives knew from the outset that cuts to tax credits would be politically hazardous. It was for this reason that they denied their existence during the general election campaign. Osborne’s pledge to cut £12bn from the welfare budget, while protecting pensioner benefits, made the £30bn tax credits bill an unavoidable target. Had the Tories formed another partnership with the Liberal Democrats, they would have modified their plans. Yet their unexpected majority of 12 forced them to implement policies that had been designed as negotiating positions.

When the Chancellor delivered the first Conservative-only Budget in 19 years, he unveiled what seemed to many an ingenious solution: a “national living wage”, which was scheduled to reach £9 an hour by 2020. Ed Miliband was among those startled by Osborne’s audacity, telling one MP that he felt “sick to [his] stomach” (Labour had promised a lower minimum of £8). But his mood changed when the Institute for Fiscal Studies (IFS) announced that three million families would lose an average of £1,000 a year due to the cuts – even after the introduction of the national living wage. It was Osborne’s refusal to acknowledge the losers, let alone to compensate them, that proved his undoing.

After an interregnum in which Jeremy Corbyn’s election as Labour leader dominated the news, an array of individuals and institutions challenged the Chancellor. Boris Johnson used his Conservative conference speech to warn, “We [must] protect the hardest working and the lowest paid.” The Tories’ London mayoral candidate, Zac Goldsmith, and the former leadership candidate David Davis also protested. To the Conservatives’ surprise, the Sun, under its combative new editor, Tony Gallagher (formerly of the Daily Telegraph and the Daily Mail), denounced the tax credit cuts as “bonkers” and demanded compensation for the low paid. Frank Field, the IFS and the Resolution Foundation, now led by Labour’s former director of policy and rebuttal, Torsten Bell, continued to produce an arsenal of statistics showing the losses that the working poor would suffer.

The dry data was given a human face on 16 October when a Question Time audience member, Michelle Dorrell, who voted Tory in May, tearfully lambasted the party for cutting tax credits “after promising you wouldn’t”. As Conservative MPs endured similar protests from their constituents, they questioned Osborne’s judgement. The newly elected MPs Heidi Allen and Johnny Mercer made Commons speeches appealing to the government not to betray “the vulnerable”. After Osborne’s concession, Mercer, a former soldier, told me: “This is how democracy works and we should celebrate it. At no stage have I ever found a minister who doesn’t want to listen to his MPs, who are reporting ultimately what they hear from constituents.” In contrast to the sharply Thatcherite 2010 intake, many of the newest Conservatives are unashamed pragmatists, less preoccupied with theory than with voters' experiences.

Davis, one of the two original Tory rebels, believes that Osborne may simply have been unaware of the losses that claimants would suffer. “It was probably a technical mistake, because they didn’t do the full impact assessment,” he told me. “The Chancellor was not conscious of what the actual impact would be on relatively impoverished people. We got too far down the pipe before it was realised what the consequences were.” Stephen McPartland, his fellow rebel, said: “Conservative MPs were concerned because they didn’t want to be in a position where they were punishing people who get up and go to work and try to do the right thing.”

Osborne now has a small window, before the Autumn Statement on 25 November, in which to calibrate his response. Tory sources suggest that he is likely to use the fiscal headroom provided by the forecast budget surplus of £10bn in 2019-20 to reduce the tax credit cuts, currently due to save £4.4bn. Conservatives privately express relief that they are facing Corbyn, rather than Yvette Cooper, who they believe would have been a far more formidable opponent.

The Chancellor’s opponents both inside and outside the Conservative Party have drawn satisfaction from his humbling. Yet Osborne is rarely as formidable as suggested in his moments of triumph, nor as feeble as suggested in his moments of defeat. He has been written off before - after the Oleg Deripaska affair, after the "omnishambles" Budget and after the "1930s" cuts - and recovered each time. His fate is not set. It will be some time, however, before he can once more joke about the “terrible” policies of his predecessors.