David Cameron has been delivered a humiliating blow, with a broken Andrew Mitchell quitting as chief whip, abandoning a month-long fight to save his career and fend off claims that he had referred to a police officer as a "pleb".

Mitchell's decision – relayed to the prime minister personally in a meeting at Chequers after Cameron returned from an EU summit in Brussels – is a huge blow to the prime minister, who had stood by him in the face of an onslaught by the opposition and the Police Federation and growing doubts among his own backbenchers.

Cameron used his authority at prime minister's questions on Wednesday to defend Mitchell from an attack by Ed Miliband and private threats to resign by the deputy chief whip, John Randall.

The meeting between Mitchell and Cameron, at 4pm on Friday, came as a separate row was breaking over whether George Osborne had travelled in a first-class train carriage on a standard ticket.

In his resignation statement, Mitchell said: "Over the last two days it has become clear to me that whatever the rights and wrongs of the matter I will not be able to fulfil my duties as we would both wish. Nor is it fair to continue to put my family and colleagues through this upsetting and damaging publicity."

He continued to defend himself from claims that he had called the police morons or plebs, claims made by the police officers in Downing Street who faced a bitter verbal volley from Mitchell.

The row developed after police refused to open the main gates to allow him to take his bicycle through, instead forcing him to use the side gate. The exact words used by Mitchell on the night have been disputed ever since, with the former chief whip in effect saying the account in the subsequently leaked police log book on the night had been inaccurate.

In his resignation letter to Cameron, Mitchell said: "I have made clear to you – and I give you my categorical assurance again – that I did not, never have and never would call a police officer a 'pleb' or a 'moron' or use any of the other pejorative descriptions attributed to me.

"The offending comment and the reason for my apology to the police was my parting remark: 'I thought you guys were supposed to f***ing help us.' It was obviously wrong of me to use such bad language and I am very sorry about it and grateful to the police officer for accepting my apology."

Cameron responded: "I was sorry to receive it, but I understand why you have reached the conclusion you have, and why you have decided to resign from the government."

The education secretary, Michael Gove, said Mitchell had suffered a moment of exasperation and he praised the prime minister's loyalty: "I think one of the many reasons why David Cameron is an outstanding leader and an outstanding prime minister is that he believes in backing those who work for him.

"He is not someone who, at the first whiff of trouble, abandons those who've pledged their loyalty to him. I think what you want in any leader is the knowledge that they will back you, that they will show steadiness, that they will do anything they can to ensure the team that's been picked stays together, works together, fights together."

In a sign of the chaos surrounding No 10, one hour after an official briefing that there would be no announcement of who would replace Mitchell as chief whip, it then emerged that Sir George Young was to replace him.

Young, 71, an old Etonian and founder member of the House of Commons bicycling club, had departed his post as leader of the Commons in the summer cabinet reshuffle, a decision he had taken with good grace.

He has previously worked in the whip's office and will bring eloquence rather than malevolence to the task of bring the increasingly exasperated backbench MPs into line.

Mitchell had hoped he could defuse the row after the police officers at Downing Street accepted his apology and said they did not wish to pursue the matter. He also held less successful contrite meetings with the Police Federation in the West Midlands. Mitchell, only appointed to the post in the September reshuffle, stayed away from the Tory party conference in Birmingham last week in the hope that the issue would subside.

But, by Thursday evening, he decided that he simply did not have the support of his backbench colleagues to impose discipline upon them. Some MPs also noticed that the normally ebullient and tough Mitchell was drained of spirit and looked to have lost weight.

He contacted the prime minister's office on Thursday and asked to see Cameron on his return to Chequers.

Mitchell's decision to quit the post he has craved for more than a decade came following a difficult meeting of the 1922 backbench committee on Wednesday, and then the further soundings he took on Thursday. He was struck by the lack of support he enjoyed among the 2010 intake of Tory MPs, although he was retaining the loyalty of those who had known him for longer.

Mitchell also sensed that he was becoming a lightning conductor for wider concerns about the Conservative party being a party of privilege and snobbery. The government already had difficult relations with the police owing to the tough reform programme that it is imposing, and the Mitchell row poisoned relations further.

It is understood that Cameron broadly knew why Mitchell wished to see him at Chequers, and carried this news with him through the EU summit. Mitchell knew controversy would continue to dog him, with Labour staging a debate in the Commons next week that would have seen an attempt to dock his pay. The international development select committee was also planning to stage an inquiry into his decision to hand a fresh round of government aid to Rwanda in one of his last acts as international development secretary, his previous cabinet posting. The Rwandan regime has been accused of fomenting violence in the Democratic Republic of the Congo.

Sir George Young: back on the front line

Sir George Young thought a month ago his life on the front line of politics had come to an amicable closure when he agreed to stand aside as leader of the house in the cabinet reshuffle.

He happily gave way at the age of 71, creating a restorative berth for Andrew Lansley, the former health secretary. Such is the unpredictability of politics, his semi-retirement lasted less than a month. It will be a familiar experience. He was sacked by Lady Thatcher only to be brought back a few years later. He is clearly someone to whom leaders turn in times of need.

For Cameron, his reappointment is doubly helpful. He calms a rocky ship and ensures there does not have to be a wider reshuffle.

On the surface, the old Etonian bicycling Baronet will look like a deeply inappropriate person to restore to an administration seeking to shed the patrician image created by Andrew Mitchell episode. But he is courteous, wry, insightful and very much on the left of his party. The quote frequently attributed to him that the homeless are what people leaving the opera have to step over was designed not to slight the homeless but ridicule the rich.

Tory MPs from all sides respect him as a House of Commons man: keen to reform parliament and reconnect it to the public. He has tried to give the e-petitions edge by saying they must once they reach a certain level of signatures be considered by a backbench committee responsible for determining the timetable of Commons debate. He has also said MPs rather than government should control more of what happens in the Commons.

But as a former deputy chief whip in the Thatcher administration, he is also versed in what makes MPs tick, and from the recent vantage point of leader of the House he will have seen the new intake of Tory MPs close up. Restoring discipline to a group of disaffected backbench MPs will not be easy, but if experience and guile counts for anything, he could succeed.

Patrick Wintour