The pace of the British withdrawal from Afghanistan could quicken next year because military commanders have changed their views about how many troops need to remain to help local security forces fight the Taliban, the defence secretary has said.

In an interview with the Guardian, Philip Hammond admitted that six months ago the military was privately pushing "for keeping force levels as high as possible for as long as possible".

But he said British military thinking was evolving because commanders had been "surprised by the extent to which they have been able to draw back and leave the Afghans to take the lion's share of the combat role".

The UK has closed 52 of its military bases and checkpoints in Helmand province over the last six months, leaving 34 still operating, he revealed.

"I think there is a bit of a rethinking going on about how many troops we do actually need … there may be some scope for a little bit more flexibility on the way we draw down, and that is something commanders on the ground are looking at very actively."

Though Hammond refused to go into details and said no decisions had been taken, this is the first acknowledgment from the government that the UK's long march out of Afghanistan could accelerate markedly next year, rather than waiting until 2014 – the end date for the British military effort in Helmand, which began in 2006.

In the interview Hammond also said:

• He was concerned that the rioting which led to the murder of the US ambassador in Libya could spread to Afghanistan, and measures had been taken to beef up security around British bases, and prevent the "unnecessary exposure of personnel" to potential trouble, especially on Friday after prayers.

• Britain would not be "spooked" by the spate of "green on blue" attacks this year – where Afghan troops killed their British colleagues. He revealed that 60 soldiers had been sacked from the Afghan army in recent weeks, and 600 others were being scrutinised by a joint UK/Afghan inquiry into the killings. He said the UK had found serious weaknesses in the way Afghan soldiers were being managed.

• Britons would have to "get used to the idea" that any peace in Afghanistan will need to involve "reaching out Northern Ireland-style to at least the moderate part of the insurgency, to try to bring it inside through reconciliation and integration".

• The Afghan government needed to do much more to bring about a political settlement with the insurgents because the diplomatic effort was lagging behind the military campaign.

During a trip to Afghanistan in which he met President Hamid Karzai in Kabul and mingled with troops at the British base in Camp Bastion, Hammond was frank about the situation in Afghanistan and what he regarded as the UK's principal objectives.

However, it is his remarks on the "draw down" that will provoke interest among all Britain's partners in Nato's International Security and Assistance Force.

Britain is withdrawing 500 troops by the end of this year, leaving 9,000.

But though the extent and timing of next year's withdrawal will depend on US plans, this is Hammond's first admission that British commanders are now reassessing how many troops they need – because, he said, they have been encouraged at the way Afghan National Security Forces have taken to the battle.

This could lead to thousands more troops coming home next year than might have been expected, a withdrawal that would probably start in September.

"I think that the message I am getting clearly from the military is that it might be possible to draw down further troops in 2013," Hammond said. "Whereas six months ago the message coming from them was that we really need to hold on to everything we have got for as long as we possibly can. I think they are seeing potentially more flexibility in the situation.

"Talking to senior commanders you get a clear sense that their view of force levels is evolving in light of their experiences."

He was also unusually blunt about Britain's role in Afghanistan. Now that al-Qaida had been "eliminated" from the country, it was not right to ask troops to put their lives at risk for nation-building, he said.

"We have to be clear why we came here in the first place. I believe very clearly that if we are going to ask British troops to put themselves in the firing line, we can only do that to protect UK vital national security interests.

"We can ask troops who are here to help build a better Afghanistan, but we cannot ask them to expose themselves to risk for those tasks. We can only ask them to expose themselves to risk for Britain's national security, which is what they signed up to do."

Hammond said the UK had "not come here to defeat the insurgency". Britain just needed to help contain the insurgency to stop terrorists getting a foothold again.

He said Britain, in an ideal world, would like to see Afghanistan become more democratic, with a society that respected human rights and improved education. But he insisted that was not the ultimate measure of success.

"The ultimate measure of success must be the extent to which we can leave Afghanistan in a state that will continue to deny its territory to international terrorists," he said.

Despite ongoing fighting in Helmand that has led to more than 20 British troops being killed in recent months, including five in "green on blue" attacks, Hammond said the security situation was improving, and that Afghan forces were now able to protect the main towns, pushing the insurgents into the desert areas.

But in a swipe at the diplomatic efforts to secure peace, Hammond said "tracking people down and removing them from the battlefield" was not the best way of finding a settlement.

"There needs to be, in my judgment, greater weight given to the high level political initiative for reconciliation. We recognise that Afghan society is such that it is pretty difficult to imagine a situation where there won't be any level of insurgency … but it is also difficult to imagine in the long run a stable prosperous and sustainable Afghanistan that has not managed to reintegrate and reconcile at least a significant part of the insurgency."

He added: "The Afghan government needs to do more and the neighbours who have influence [Pakistan] also need to maintain pressure on those parties … to come to the table. We have to provide them with expert support. They want to do it, but turning a wish into reality is a complex process.

"I think most of us feel that the political process is not moving as fast as the military process and we would like to see political progress keeping pace with military progress."

Accommodating the Taliban in the peace and reconciliation process was vital, he said.

"Look at our own history. Every counter-insurgency war we have fought in post-colonial history has ended up with an accommodation with at least part of the insurgency movement.

"That is the reality. You cannot create a lasting settlement with … a significant part of the population locked out. You have to bring moderates into the process to lock out the extremists as we have done in Northern Ireland."

In July three British troops were killed by an Afghan colleague in one of a series of "green on blue" attacks against Isaf forces this year. More than 40 Nato troops have died since January by these insider attacks, which is having a debilitating affect on morale – and has prompted France to declare it is withdrawing troops early.

Hammond said he had spoken to Karzai and told him that "while we [the British government] might not be spooked by this, public opinion feels very strongly about these green on blue incidents and it is absolutely essential that this problem is put back in its box".

The defence secretary said he had been "talking to one of the senior officers about one of the green on blue incidents, and the investigation has thrown up serious weaknesses in the way that some elements of Afghan troops have been managed in the past".

He added: "The Afghans are moving to address those problems and we are all over it. Where we have pointed the finger at people, the Afghans haven't hesitated to arrest them and check them out."

He said it might not be possible to eliminate the green on blue threat completely, but he believed the measures that had been taken would have an effect.

The conflict in Afghanistan has led to the deaths of 427 British soldiers, and more than 2,000 American troops. The United Nations said that 3,099 Afghan civilians were killed in the first six months of this year alone.

Hammond insisted that it had been right to set an end-date to the military campaign because if "you don't set a deadline you'll never get the job done".

While critics of the Afghan conflict have focused on the never-ending fighting with an insurgency that is weakened, but by no means defeated, and the corruption scandals that have plagued President Karzai's government, Hammond said the UK's involvement in Helmand would be regarded as a success story.

"The message for people back home is that over the years we have contained the threat to UK security, we have built the Afghan security forces to the point that they can take over that burden.

"We are confident we will leave in place a solution that will not look perfect … but if it maintains the integrity of Afghanistan and stops the use of it for international terrorists, we have achieved our primary purpose."

He added: "Even if we had achieved nothing lasting, every year that goes by keeping the bombers at bay, keeping them off our streets, is a significant achievement in itself. But we have clearly built the basics of a future that will deny the space of Afghanistan to those who would seek to harm us."