Justice secretary says government has no intention of changing law and problem will not be solved by decriminalisation

Britain is "plainly losing the war on drugs" and may even be going backwards, the justice secretary, Ken Clarke, has acknowledged to MPs.

But he insisted this was no reason for despair and politicians had to keep on trying to curb a problem that would not be solved by decriminalisation.

Clarke was giving evidence to the Commons home affairs select committee inquiry into drugs, before which Richard Branson, Russell Brand and other high-profile figures have made the case for reforming Britain's 40-year-old drug laws.

The justice secretary made clear to MPs on Tuesday that "the government has no intention whatsoever of changing the criminal law on drugs", adding he was strongly opposed to decriminalisation as it would encourage more youngsters to experiment.

Clarke had been invited by the Tory MP James Clappinson to give the committee the benefit of his experience in government on how he thought the drugs problem ought to be tackled.

"I have not reached the stage of that blinding insight about exactly how we are going to improve our record, is the honest truth," he said. "We have been engaged in a war on drugs for more than 30 years. We are plainly losing it. We have not achieved very much progress. The same problems come round and round.

"But I do not despair – we keep trying every method we can to get on top of what's one of the worst social problems for the country and the biggest single cause of crime."

He acknowledged the existing criminal law was not working.

"We are all disappointed by the fact that, far from making progress, it could be argued we are going backwards at times. But my own personal view is that I would be worried about losing the deterrent effect of criminalisation of youngsters who start experimenting … One thing that does put them off is that they would get into trouble with the police."

The justice secretary resisted the committee's suggestions that the government might need to appoint a "drugs tsar" to co-ordinate the approach across Whitehall, saying that departments now worked together on the drugs strategy much more smoothly than they used to.

He recalled when he tried as health secretary in the late 1980s to get some co-ordinated effort to tackle the drug problem but said his ministerial colleagues had not "taken the blindest bit of notice" of him.