The government is to forge a "leaner, more efficient state" on a permanent basis, David Cameron has said as he signalled he had no intention of resuming spending once the structural deficit has been eliminated, a clear change to claims made after the last general election .

In a change of tack from saying in 2010 that he was imposing cuts out of necessity, rather than from "some ideological zeal", the prime minister told the Lord Mayor's banquet that the government has shown in the last three years that better services can be delivered with lower spending.

Cameron said that the government would press ahead with tackling the deficit after cutting it by a third. But he made clear that his party intended to go further.

"We are sticking to the task. But that doesn't just mean making difficult decisions on public spending. It also means something more profound. It means building a leaner, more efficient state. We need to do more with less. Not just now, but permanently."

The PM cited Michael Gove's work in cutting administrative staff at the education department by 40% while 3,000 free schools and academies have been established. He also said the government has cut 23,000 administrative posts from the NHS while employing 5,000 more doctors.

He said: "So you can have a leaner, more efficient, more affordable state that actually delivers better results for the taxpayer."

The remarks by the PM contrasted with his claim after the 2010 election. In his New Year's message for 2011, issued on 31 December 2010, he said: "I didn't come into politics to make cuts. Neither did Nick Clegg. But in the end politics is about national interest, not personal political agendas.

"We're tackling the deficit because we have to – not out of some ideological zeal. This is a government led by people with a practical desire to sort out this country's problems, not by ideology."

A few months earlier that year, in his first Tory conference speech as PM, Cameron said he would have preferred to tackle the deficit in ways other than public spending cuts. He said: "Everyone knows that this government is undertaking a programme of spending cuts. I know how anxious people are. 'Yes', they say. 'Of course we need to cut spending. But do we have to cut now, and by this much? Isn't there another way?'

"I wish there was another way. I wish there was an easier way. But I tell you: there is no other responsible way. Back in May, we inherited public finances that can only be described as catastrophic."

In a sign the PM believes that he needs to make a more aggressive defence of spending cuts, Cameron also rebuked the Archbishop of Canterbury, who warned earlier this year that children would "pay the price" for the government's decision to cap benefits at 1% a year until 2016. In remarks endorsing a letter by 43 Anglican bishops, issued a few weeks before his enthronement, Welby said in March of the benefits up-rating bill: "These changes will mean it is children and families who will pay the price for high inflation, rather than the government."

Cameron said: "There are some people who seem to think that the way you reduce the cost of living in this country is for the state to spend more and more taxpayers' money. It's as if somehow you measure the compassion of the government by the amount of other people's money it can spend.

"At a time when family budgets are tight, it is really worth remembering that this spending comes out of the pockets of the same taxpayers whose living standards we want to see improve. I hope the Archbishop of Canterbury will forgive me for saying – it's not robbing Peter to pay Paul – but rather robbing Peter to pay Peter. Let's be clear. The single biggest threat to the cost of living in this country is if our budget deficit and debts get out of control again."

Cameron's remarks were also aimed at Ed Miliband, who has dominate the political agenda since the conference season with his focus on the costs of living.

The prime minister also used his speech to announce that he would lead a trade mission to China in December. He had to postpone a visit earlier this year after China, which was irritated with Cameron after he met the Dalai Lama, made clear that he would not meet the main Communist leadership.

He said: "As China's new leadership sets its direction for the next 10 years, as their country's star continues to rise in the world, I will take senior British ministers – as well as business leaders from every sector large and small – to forge a relationship that will benefit both our countries and bring real rewards for our peoples. Opening the way for British companies to benefit from China's vast and varied markets and preparing the way for a new level of Chinese investment into the UK.

"This is a relationship that is for the long term, that matters for Britain and China, and which I look forward to continuing to strengthen in the months and years to come."