Prison reformers today welcomed what appeared to be a major shift in the approach to penal policy outlined by the new justice secretary, Ken Clarke, over the weekend.

The lord chancellor questioned why the prison population – at 85,000 – was nearly double what it was when he was home secretary in the early 1990s.

Clarke confirmed that he is looking for cuts in the £2.2bn prison budget and seemed to indicate that he did not regard short prison sentences as effective in cutting reoffending rates.

He acknowledged that members of the public were still "very, very worried about lawlessness" but said that their "fear of crime" is probably out of proportion to what they actually face.

"It's not to be soft on sentencing, it's to be sensible on sentencing, and bear in mind everybody who is sent to prison costs more than it costs to send a boy to Eton. So, all right, I'm all in favour of spending it when it's effective and justified, and that we will do. And we're looking at sentencing, not starting just from let's have more people in prison, let's have fewer people in prison … but what actually works, because the public are still very, very worried about lawlessness," the justice secretary said in a Sky News interview yesterday.

"What I'm looking at is, in this case, sentencing. Our first duty is to protect the public – there are some very dangerous and nasty people that need to be in prison," said Clarke.

"But why is the prison population twice what it was when I was the home secretary not so very long ago?"

The Conservatives went into the election pledging to match Labour's plans to build sufficient prisons to house 96,000 by 2014. The Liberal Democrats had a pledge to halt the prison building programme and urge the courts to use community punishments instead of short prison sentences. The coalition agreement split the difference by agreeing to take a fundamental look at sentencing policy, which the justice secretary outlined yesterday.

Clarke confirmed that he had to look for cuts in the justice ministry budget but he was to keen to take a more fundamental look than simply "salami slicing" budgets and saving a "bit of money here, and a bit of money there".

The move was welcomed by prison reformers. Jon Collins of the Criminal Justice Alliance said his comments were a welcome step forward: "At long last a politician is facing up to the unsustainable cost of our prison system. We simply cannot afford to keep building endless new prisons as more and more people are sent to prison at huge cost to the taxpayer," said Collins.

"Since Ken Clarke was last in charge of the justice system, the prison population has nearly doubled, warehousing thousands of people who could be dealt with more cheaply and more effectively in the community. Ken Clarke is right, keeping more of these people out of custody would save money and free up space and resources in the prison system to better rehabilitate those people who do need to be there."

But Conservative rightwingers disagreed: "It's very sad that somebody of Ken Clarke's calibre is talking such drivel. This is a ridiculous false economy – it saves money to have the most persistent criminals in prison," said Conservative MP Philip Davies.

"If short sentences do not work, the argument should be for longer sentences, not putting them out on the streets to terrorise communities."

The prison population stood at fewer than 45,000 when Clarke was home secretary in 1992-3.