After many critical periods over the past decade, Afghans now face their most important yet – and the problems are huge

The building used to be the Russian Officers' Club, and its position on top of a hill 10 miles to the west of Kabul would have given them the best possible view of the capital that their forces captured more than 30 years ago.

It is derelict and empty now, but the view still tells a story about the country past and present; just beneath it is a newly built barracks for the Afghan National Army, where more than 1,000 trainees are put through combat exercises by Afghan and British instructors.

Beyond it are the bombed and burned out relics of the former royal palaces, and then the capital itself. The population has grown to an estimated 4 million in recent years – up by a quarter – and beneath the relentless chaos on the roads and in the markets, there are currents that suggest that a degree of confidence is returning to the people.

Watching over all this activity are the mountains of the Hindu Kush, whose peaks loom on the horizon like a set of giant sharks' teeth.

Kabul is a long way from where most of the fighting will take place this summer, but it is where the architecture of the new Afghanistan is being hastily designed by ministers, generals, diplomats and aid workers.

In recent months there has been frustration among coalition partners, particularly in the military, that the progress that has been made over the last year has not been properly recognised.

That has been supplemented in recent weeks by a more general concern – which became acute with the death of Osama bin Laden – about the commitment of the international community in the years ahead.

Corruption

How the coalition defines success in Afghanistan is changing all the time. Expectations are being managed about what the country will look like, with warnings that many in the west may not like what they see.

Nobody in Nato talks about being able to defeat the Taliban any more. The best that can be hoped for is that the political and security situation can be made good enough to give the Afghan government a chance to go it alone after 2014. But nobody believes the country will be ready to stand on its own feet through the transition phase and thereafter without considerable support.

Who will give that help, how long will it be needed, and who will pay?

Afghanistan remains one of the most corrupt countries in the world. It has a life expectancy of 44 and only 12% of women are literate. None of those problems will be solved in three years. "This is a developing country, starting from a very low base, with shooting attached," said one western official. Unlike Iraq, "it doesn't have an educated middle class and it doesn't have oil to pay for everything". The British ambassador in Kabul, Sir William Patey, says it will take two generations for Afghans to see real transformation. Some aid workers estimate it is a 25-year project.

The civilians are looking to people such as General James Bucknall, a British Coldstream Guards officer who is second in command of the International Security and Assistance Force (Isaf).

Bucknall knows – as do they – that real progress can only be achieved if the country is not in a state of permanent conflict.

In his office on the first floor of Isaf's heavily fortified headquarters in Kabul, Bucknall, a veteran of Iraq, Northern Ireland and the Balkans, concedes that this is "the most complex and demanding theatre I have ever worked in".

But he sets out why he thinks a corner has now been turned, nodding to the surge in American troop numbers that has made it possible.

"We have halted the insurgents' momentum. And in some areas where we have really applied resources we have regained the initiative. We have successfully removed a number of safe havens in Afghanistan, some of which the insurgents have held for a long time, particularly around Kandahar. We have also removed substantial munitions, far greater than we ever have before."

Special forces operations, he says, are being conducted on a "nightly basis" against mid-level insurgent leaders. "Those insurgents that went away for the winter are coming back to a changed environment. I think that in certain cases they are finding that communities are more prepared to stand up and reject them than they have before.

"The key is now to make those gains, hang on to them and expand them where we can and make them irreversible. The people of Afghanistan have tasted the Taliban before and I don't think there is any evidence that they enjoyed what they experienced."

Bucknall argues that it is unfair to judge the military campaign against the Taliban over the decade that coalition forces have been here. "There is this narrative that we have been at it for 10 years. As I say, we haven't. We have only really been playing this sensibly, or properly, with the right resources, from last year."

He will not readily admit that mistakes have been made, though he comes very close. "I think that it would be … not credible for us to say that we have got this campaign right from the start. We clearly haven't. Patience is important. We have to earn that patience. We have asked a lot of the British people. We have committed an enormous amount to this campaign. What we have to do is show that we are making progress, which we are. Now is not the time to blink."

Even if Afghan security forces take over the fight against the insurgency in 2014, Bucknall warns that British troops will be here for "many years". He likens the situation to Iraq, Saudi Arabia and other countries where UK forces still have a presence.

"I expect us still to be training the Afghan national security forces, to be mentoring and advising them in their ministries and having a commitment that goes some time beyond that.

"How long that is, we will work out in due course.

"It will be years – absolutely, yes. That signal of long-term commitment is an essential part of our short-term success of this campaign. There is a lot at stake – not just our national security, but our influence in a region of strategic importance to this country."

As Bucknall continues overseeing the fighting, an American general – William Caldwell – has responsibility for getting the Afghan Security Forces (ASF) in shape for the handover.

Caldwell has been in Afghanistan since autumn 2009 and was given a job that he initially thought was impossible. The problems went far deeper than he had imagined, and he was so horrified he was prepared to break military etiquette to say so – first to the late Richard Holbrooke, who was then America's special envoy to Pakistan and Afghanistan, and then to General Stanley McChrystal, the former overall commander of Isaf.

"Mr Holbrooke said to me, 'you've got to do something about literacy, General Caldwell.' I looked at him and said, with all due respect, I am a military man. I don't do literacy. I'm not running a nation-building exercise."

Within 60 days, Caldwell says, he was eating his words.

Only 14% of recruits to the Afghan army and police are literate. "I can't even begin to tell you what that means," he says. "I couldn't comprehend it until I got here on the ground. I told my guys, if we don't instil some literacy into the force, it will never be self-sustaining. Without literacy, they can't even account for their equipment because they cannot read the serial numbers. How can they call in air strikes, artillery fire? We have bought them 45,000 vehicles already, 56 airframes. We are buying a lot of equipment that requires basic reading to do maintenance on them."

Caldwell recalls one incident that sums it up. "[There was] a very sad story of an Afghan unit out on a military operation. They had all the proper equipment. They rang in and said 'we need a medivac'. We said, where are you, give us the co-ordinates. They started describing the place. Nobody could read a map in the entire company."

With an initial target of recruiting and training a force of 305,000, Caldwell started a huge literacy programme to run alongside basic military training. He also told his boss to provide more professional military trainers, or risk putting the whole enterprise in jeopardy.

"This is mission impossible. In fact I wrote that in December of 2009 to McChrystal. I sent him a classified memo and said I cannot accomplish this mission with the resources you have currently allocated me. It's impossible – you can't do it."

Officer class

Since then, the number of trainers under his command has quadrupled to 4,000, and the target for security force recruitment has been reached. Last year, Caldwell hired 100 teachers. Now he has 2,100. New recruits do two hours' reading and writing every day during eight weeks' basic training. Those likely to rise up the ranks are given extra tuition.

"On any given day we now have 34,000 in training programmes in Afghanistan. The magnitude is almost overwhelming," says Caldwell.

"When we started here, if you wanted to be a soldier in the Afghan army, as long as you were there the day you started, and the day you graduated, you graduated. Could you shoot your weapon, accurately? It didn't matter."

An Afghan officer class is slowly emerging, with 600 being taken on as cadets at a national military academy this year. Up to 4,600 had applied for the posts. "We are really pushing them, driving these kids hard," said Caldwell. "We are not going to allow them to progressively learn, we are going to leap them forward to the 21st century. We've given every one of them a laptop. Most of these kids have never driven a car, they may not know how to flush a toilet."

President Hamid Karzai had hoped to increase the total number for the ASF to 378,000, but the US has said it will only fund an increase up to 352,000.

"We have built an army and police force that still needs help," Caldwell explains. "If we don't grow beyond 305,000 there will still be a dependence on Nato military forces – 352,000 will do what is necessary."

Inevitably, the growth of the Afghan military has not been free of problems. There have been 20 incidents in the last six years when insurgents have managed to breach the security checks and embed themselves in the Afghan army, causing havoc – three British soldiers from the Royal Gurkhas were killed last year by one such insurgent.

The US has trained a covert force of Afghan counter-intelligence officers to sniff out infiltrators and impersonators. Keeping the police and army ethnically balanced has also proved difficult, but the ratio is roughly right, though they would like more Pashtuns from the south.

The Afghan army is made up of kandaks (battalions). One of them – the 2nd battalion second brigade of the 205th corps – has just been declared the first to be totally independent of coalition forces. Another 157 kandaks need to reach that standard by 2014 for Nato troops to be able to withdraw from all combat duties.

One way to shortcut the fighting would be for the Taliban to negotiate a settlement of some kind with the Afghan government. Efforts are being made through different ministries, foreign intelligence agencies and even the UN to make contact with leaders of the movement. There are varying descriptions of how successful these efforts have been; some officials say they haven't even reached talks about talks, but others are more optimistic.

Tribal leaders

Turkey is emerging with a potentially pivotal role; one idea now gathering momentum is for the Taliban to have a political base there, allowing face to face talks to take place about how a resolution can be found. The initiative, which has the support of Karzai, is partly tactical: the Taliban leadership would probably refuse to conduct negotiations in Afghanistan or elsewhere in the region for safety reasons. It would be more difficult to refuse talks so far away from the fighting.

Karzai has problems too. The tribal leaders in the northern provinces who fought against Taliban rule are against any move that gives the movement political legitimacy or, worse still, offers it the appearance of being a government in exile. The death of Bin Laden has, however, given some political and military leaders in Kabul the hope that Mullah Omar, the Taliban leader, will separate from al-Qaida – one of the key requirements for any kind of settlement.

"The message to the Taliban is that there is an opportunity to come in from the cold," said Patey, the British ambassador. "You have to abandon international terrorism, abandon your former allies in al-Qaida, work within a constitutional framework, lay down your arms. Peace is open."

Patey is also pragmatic about what support the Taliban may have in the country after 2014. "The Afghans will be in charge. And that will be frustrating for some people in Europe because things might happen here that they won't like. This is an Islamically conservative country and will remain Islamically conservative. It will have a value system different from ours."

There is hope that Pakistan could help bring the Taliban leadership to negotiations. Like Bin Laden, Mullar Omah is assumed to be hiding there, somewhere on the border between the two countries. If the Pakistani government were to make it clear that his presence in the country was no longer acceptable, that might push him into diplomacy. That's what the optimists believe, anyway.

They also hope Omar's hand may be forced by the number of insurgents who appear to be giving up the fight. Nine months ago, the Afghans started a reconciliation and reintegration process, offering insurgents the chance to lay down their weapons and go back to their communities with honour intact. In the first seven months, 700 fighters and some of their commanders signed up. In the past six weeks, another 700 have applied. Two thousand more have registered an interest by filling out the forms.

"As the fighting season has started, the number of people formally in the programme has doubled, and the expressions of interest from communities, elders, and fighting groups themselves has quadrupled," said General Phil Jones, the British director of Isaf's force reintegration cell. "People are voting with their feet in places like Kandahar and Helmand."

Those who join the process have a three-month cooling off period during which they are paid $120 (about £75) a month while their communities work out how best to bring them home. The rules have been made deliberately vague so that local elders and tribal leaders can decide how best this is done, and whether any crimes they have committed should be taken into consideration. "Thus far, the vast majority [of those who have signed up] are low-level fighting groups that have been bruised into insurgency over recent years and have become affiliated with the Talibs," said Jones.

"They have become prey for the more fundamentalist networks, supporting them with intelligence, planting improvised explosive devices (IEDs)," he said. "They don't necessarily think of themselves as ideological fundamentalists who are fighting against the government. This is the nature of much of the insurgency.

"In years past, either economic or political exclusion, or poor governance, or predatory behaviour by the police years ago – these sort of issues forced them to defend themselves. Life has changed and this gives them an option to step out."

He added: "One would hope that as the dust settles, there would be a much stronger desire to reach out and accelerate the process of peace amongst the senior leadership of the Taliban. There is already is an increasingly vibrant dialogue – can that solidify into something like a peace process?"

Until that happens, the coalition will continue to try to rush Afghanistan into political and military self-sufficiency. In July, seven of the country's 34 provinces will become self-governing. In another six months, seven more will attempt to stand on their own. The hope is that people will start to trust the new institutions and leaders.

Simon Gass, the former British ambassador in Iran who is now Nato's senior civilian representative in Afghanistan, said people should not overburden transition with unrealistic expectations.

"We hope that we can show improvements, and make sure that there is a level of confidence in basic services that will encourage the support of the institutions of governance. Transition does not mean that we can solve all Afghanistan's problems," he said.

The quality of provincial governors had improved, said Gass, but there were "still real problems of capacity beneath them". Corruption remains a huge issue at all levels of society and in the police. The idea that Afghanistan will be free of corruption by handover is deemed ridiculous, though concerted efforts are being made. I don't expect Afghanistan in 2015 to be a paragon of virtue, free of corruption with the best governance," said Patey. "But I do expect it to be better and I expect the Afghans to have built some institutions that can deal with corruption."

Critical period

Although Afghanistan has had many critical periods over the last decade, it faces perhaps the most important yet. This summer's fighting season should show whether the military surge has been as successful as Bucknall hopes.

Within weeks, a number of Afghan provinces will be running themselves. The US and Afghanistan are negotiating a strategic partnership that will inform how America sees its commitment to the country in the second half of the decade; and in July President Barack Obama will set out how many American troops are to withdraw from the country this year.

Some observers say too much is being done too soon, that the timeline is bogus, and that the new institutions are being overburdened.

Isaf generals and diplomats recognise that there is a fear among the population that they will be abandoned after 2014, but nobody knows what will happen for sure yet.

The problems are considerable, and across a number of fronts, but the price of failure is huge.

"The Afghans have not just looked into the abyss, they have been in the abyss," said General John Nicholson, deputy chief of staff of operations in Isaf. "They are the toughest people I have ever met."