DOWN a rough track, glowing silver under the moon, rolls a black pick-up truck carrying a 500lb bomb. The white walls of the American army base loom, but no shout or shot comes. Your correspondent fingers his detonator.

Unchallenged, the truck drives alongside the base, then we blow ourselves up. The first blast, in a yellow flash, lights up a guard-tower and the anxious face of a young GI. The second, after we bombers have scrambled for safety, is much bigger—a hollow boom and an explosion of orange fire that soars 100 feet into the night sky, lighting up skinny pines all around. As the flames fall, The Economist's fellow suicide-bomber shouts to the sentry: “Go tell your buddies, you're all dead.”

That should teach them not to leave open the approach to their perimeter, a mistake that cost many American lives in the early stages of Iraq's insurgency. But these soldiers will live. Their white walls are chipboard, the bomb is a “reduced-blast” special effect, and although fictionally in the Afghan region of “Talatha”, they are at Fort Polk in southern Louisiana, 200,000 acres of pine forest which the army uses for training. The soldiers, members of a 3,500-strong infantry brigade in the final stage of preparation for a mission to Afghanistan, will be extracted from the woods for 24 hours and then, chastened, reinserted to fight again.

Car bombs are not the only bit of Iraqi-Afghan verisimilitude the brigade experienced at Fort Polk's Joint Readiness Training Centre (JRTC) last month. Attacks with simulated roadside bombs (known as improvised explosive devices, or IEDs), rockets, mortars, rocket-propelled grenades (RPGs) and small arms, using special effects and lasers, are unrelenting.

The assailants—160 American soldiers dedicated to the task, and dressed accordingly—come in two forms: al-Qaeda terrorists, based in an off-limits bit of the wood called Pakistan, and Taliban insurgents living in 18 mock villages. Another 800 role-players live with them, acting as western aid workers, journalists, peacekeepers, Afghan mayors, mullahs, policemen, doctors and opium farmers, all with fake names, histories and characters. Some 200 bored-looking Afghan-Americans are augmented by local Louisianans in Afghan garb. A clutch of Vietnam-veterans with missing limbs, splashed with fake blood, make terrific bomb victims.

Fort Polk has seen huge changes in the past two years. Designed for light infantry and special-forces troops, it has always dealt with some parts of guerrilla warfare, such as booby-traps and RPG attacks. But in the past the “insurgents” wore blue armbands to distinguish themselves, a tactic strangely shunned by America's enemies in Iraq and Afghanistan. There also used to be no more than 50 civilian role-players on the battlefield.

The changes are expensive—the basic cost per brigade of a month at the JRTC has gone up from $2m to $9m. And similar changes are under way at the army's two other Combat Training Centres (CTCs), where the army simulates battalion- and brigade-sized battles. Fort Irwin, California, used to be dedicated to tank battles. Two years ago, not a single building dotted its 600,000 acres of desert. Now there are a dozen mock villages and plans for a $50m mock city. Two Hollywood companies have been hired to improve the army's flashes and bangs, and to give acting classes to the role-players.

A catalogue of blunders

Stung by its setbacks in Iraq and Afghanistan, the American army is trying to change. The challenge is immense, and the army's recent past suggests reasons for cynicism.

Since quitting Vietnam in 1973, America has fought dozens of counter-insurgencies and small wars. But most of those, in Central America, were mainly special-forces operations. And in places like Somalia and Haiti it often made the same blunders as against the Viet Cong. If the army's rules of engagement were stricter, there were still too many strikes on innocent targets, too much use of firepower and too little effort to understand the culture. Faced with a counter-insurgency, the American army's preferred tactic was attrition.

Of course, all wars are different. No previous campaign could have provided a blueprint for success in Iraq. Suicide-bombers are a different threat from Marxist guerrillas. But history can suggest a few solid principles for successful campaigns against insurgents; and too often, in Iraq, American troops have ignored them. A pre-Iraq comparison between the American and the British armies is instructive.

In their routine planning and training, the British expected to find civilians on their battlefield; the Americans did not. The British taught the virtue of restraint, to limit civilian casualties and the strategic damage they cause. American soldiers were trained to wipe the enemy out. British soldiers were trained in crowd control and basic forensic skills; American soldiers rarely were. In April 2003, nervous American soldiers fired into a crowd of protesters in Fallujah, killing and maiming scores. Within weeks, the Iraqi town had risen against the occupation, culminating in two terrible battles last year.

In more peaceable southern Iraq, meanwhile, the British acted on their training. Their first aim was to win the civilian population's trust. One way was through information operations (IO), which means, at the crudest level, generating good public relations for the army. “The Brits do this as a matter of course; they had a much finer appreciation of the culture in Iraq,” says Lieutenant-Colonel Chuck Eassa, deputy-chief of IO at Fort Leavenworth in Kansas, the home of the army staff college and other cerebral institutions. For the American army in Iraq, he says, IO was a “low-density skill set”. Each division of 19,000 soldiers had only two IO officers.

Giving grief, Iraqi style

Another way to win trust is the British habit of doffing helmets and patrolling on foot. This is not always possible: when a Shia insurgency flared in southern Iraq last year, the British hunkered down behind armour and killed wave upon wave of fanatics. But when the violence ebbed, they had the flexibility to revert to friendlier tactics. To see a British sergeant in southern Iraq dismount from a Land Rover fully kitted for war, assess the local mood, then call, “Right lads, hard hats off,” is impressive. According to Lieutenant-General Sir John Kiszely, formerly deputy-chief of coalition forces in Iraq and now head of the British Defence Academy: “In future, probably nine out of ten operations will not be purely war-fighting. The soldier of the future will have to be a warrior—but much more as well.”

In part, the American army's unreadiness for dealing with civilians stemmed from a reliance on signal, not human, intelligence. As a battle raged in Iraq's northern capital of Mosul last year, shortly after its police force had collapsed, an American lieutenant-colonel showed this correspondent his control room. On a cluster of computer screens, he could see, in real-time, where each of his Stryker armoured vehicles was and what it was seeing. “It's like Sun Tzu said,” he enthused, recalling an ancient Chinese military strategist, “to win a war you need three things: to see the battlefield, to see yourself...so now I guess I just need to see the enemy.” If the enemy is hidden among the general population, technology can help, but it will not complete the task.

In a recent comparison of the British and American approaches, “Counter-insurgency lessons from Malaya and Vietnam,” John Nagl, a retired American marine officer, argues that Britain's relative expertise was not merely the result of its imperial history. In Vietnam, after all, America had ample experience of acting as an occupying power. Moreover, in both Malaya and Northern Ireland, the British were at first heavy-handed and unsuccessful. The difference, says Mr Nagl, is that the small British army was able to learn from its mistakes and change, while the bigger American army was not. Can it really be doing any better in Afghanistan and Iraq?

Tips from the ground

The signs are mixed, but encouraging. Take, for instance, the way the army is involving all its soldiers in setting TTPs—the tactics, techniques and procedures that define how it acts on the battlefield. The idea is that every TTP—be it the appearance of a new sort of bomb in Baghdad or a new way to disable one—holds lessons for army doctrine, training, appropriations, research and development and so forth.

Every brigade in Iraq and Afghanistan now has a secure intranet page, which soldiers are encouraged to fill with observations and queries. Early this year a secure online chat-room, the Battle Command Knowledge System, appeared. Besides circulating thousands of tactical questions and answers, it can help soldiers find technical experts, learn foreign languages, contact counterparts in the war zone, or squint through the web camera of an armoured vehicle in Iraq.

At Fort Leavenworth, the rather wonderfully named Centre for Army Lessons Learned (CALL) has catalogued 6,200 battlefield and training-ground observations in the past four years and produced 400 reports on them. Its staff has tripled. “They've been pumped full of steroids,” says Lieutenant-General David Petraeus, Fort Leavenworth's commander, who has recently returned from Iraq.

Cynics spot some familiar aspects to all this. Although the Pentagon is spending a fortune on bureaucratic processes, some think it remains weak on analysis and incapable of self-criticism. “Lessons Learned” is, after all, not the same as lessons learned—as CALL's boss, Colonel Larry Saul, admits. Mr Saul says a key lesson from Iraq and Afghanistan is that American soldiers must know a smattering of the local language. When assured that almost no American soldier knows six words of Arabic, Dari or Pushtu, he admits: “I can take a horse to water but I can't make him drink.”

The evidence is indeed mixed, but there are many signs of progress, even in Iraq. The American army has trained more than 100 Iraqi battalions in little over a year; fighting alongside them, it has come to understand their culture better. It is also more restrained; atrocities like the killing in Fallujah would be less likely now.

In an example recently praised by George Bush, the 3rd Armoured Cavalry Regiment carried out a model attack on insurgents in the northern Iraqi town of Tal Afar in September, first emptying it of civilians, then giving Iraqi soldiers the lead in the offensive. This success was, in large part, a response to failure, according to the regiment's commander, Colonel H.R. McMaster. On a previous deployment to Iraq, in western Anbar province, the unit had done terrible damage with random and aggressive house-to-house searches, mass arrests and thuggish crowd control. Since arriving in Tal Afar last April it had patrolled often and on foot, looking for human intelligence, and had avoided making indiscriminate arrests.

But Colonel McMaster was relatively lucky. He had seasoned troops, and enough of them, to control his area. The marine division currently in Anbar, Iraq's most violent province, is too thinly spread and too shot-at to develop such nice behaviour. Unable to get control of the rebellious towns in the Euphrates river valley, often supported by air strikes but rarely by Iraqi troops, the marines are fighting a mid-intensity war.

There are other impediments to the army's improving counter-insurgency skills. One is the dismal record of many civilian agencies in Iraq. Some $20 billion has been spent on restoring Iraq's basic services, but with little noticeable improvement—hardly the way to win hearts and minds. Another snag is the army's fixation with self-protection. In Mosul, your correspondent strolled through a souk with a dynamic and thoughtful American captain who chatted and waved to the shoppers, trying to win trust. But with his face hidden behind helmet, headset and mirrored-shades, women and children saw him and fled.

Changing an army's approach takes time. The warrior spirit, as Americans call their propensity for macho soldiering, or killing people, is deep in their military culture. In Afghanistan's violent Helmand province, an American special-forces captain—with broad experience of counter-insurgency—analysed his furtive Taliban enemies thus: “They're cowards. Why don't they step up and fight like men?” Apparently, he had not considered how he might fight if he had no armour, no radio, an ancient rifle and the sure knowledge that if he fought like a man, he would be obliterated in minutes.

Revising doctrine

Off the battlefield, the army's lesson-learning is easier to measure. At Fort Leavenworth, big changes have been made. The 1976 edition of “Operations”, the standard manual of best practice in warfare, did not mention counter-insurgency, and the army's dedicated counter-insurgency doctrine manual was last rewritten in 1965. But the section of the current edition of “Operations” dealing with counter-insurgency is now being revised, and a new version of the counter-insurgency manual, written with British help, was distributed as an interim draft on October 1st.

Next time, it's real

According to the new doctrine's authors, at Leavenworth's Combined Arms Doctrine Directorate, key changes will stress the strategic importance of civilian populations. The army's basic doctrinal aim is to move from high-intensity offensive to low-intensity security and stability operations—from creating shock and awe to winning hearts and minds. The new doctrine will emphasise that, where possible, the two levels should co-exist—where possible, for example, the army should exercise restraint in its offensive operations. In addition, the revised “Operations” will stress the need to protect civilians from external threats (such as Islamic terrorists) and internal disorder (such as looters), while ensuring the provision of “central services”, including power and water.

The interim counter-insurgency manual also reflects recent failings. It suggests ways to stop a conventional war turning into an insurgency. One way is to avoid upsetting civilians, among whom insurgents can hide, by being more sensitive to ethnic, tribal and religious differences.

Other corners of Fort Leavenworth bustle with change. A “Red University” has been established to teach “red teaming”, or second-guessing enemies and allies; it will accept its first students in January. A “red team”, dedicated to predicting future threats and to replicating them in training, has been doubled since 2002. Courses in IO have been increased from two to ten weeks. At the army staff college and a more select school for army planners, the School for Advanced Military Studies, the biggest change is that 45% of the students have recent combat experience. This fully professional army is much likelier to remember its lessons than the conscript army that withdrew from Vietnam.

But it is at the CTCs, including that bomb-blasted patch of Louisiana pine woods, that the army's effort to change its ways is most visible. Two years ago, with few civilian role-players on the mock battlefield, the soldiers' main objective was to bypass them and kill the enemy. Now most training involves civilian role-players. Even when launching an attack, the soldiers must bear in mind the overarching need for stability.

According to Brigadier-General Michael Barbero, Fort Polk's commander, “We're working hard at instilling shades of grey, at teaching commanders not to just kick in doors, to get among the population and gather intelligence.” This change, incidentally, makes the CTCs extremely similar to the British army's Operational Training Advisory Group, which has used role-playing for two decades. The biggest difference is money. American soldiers at the CTCs are shot with lasers and Hollywood-quality special effects; in mock battles at Tin City, a fake town in the Kentish countryside, unlucky British soldiers are ignited with petrol bombs.

A favourite buzz-phrase at Fort Polk is “consequence management”, or weighing the goal of an operation against its other results. During daily meetings, the training officers discuss, often in staggering detail, the brigade's latest actions, and plan their reactions accordingly. Thus, for example, a company commander has failed to deliver medicines to a village clinic, run by a “Dr Jihad”, as promised. In response, a “journalist” is commissioned to report the story for the Talatha Times. This prompts al-Qaeda fighters to deliver medical supplies to the clinic, and, in the process, they forge ties with the local Taliban. The two groups collaborate in a fierce attack on the offending American company, using car bombs, IEDs and indirect and direct fire, killing and wounding dozens. That will teach them to keep their promises.

It is unclear how long the CTCs will keep their lavish resources. Since early 2003, every training rotation has been geared to preparing a brigade or battalion for a mission in Iraq or Afghanistan. As America withdraws its troops from Iraq—whenever that will be—non-mission-specific training will resume, and the training budget will be cut. But Brigadier-General Barbero says he hopes to retain enough cash for a decent number of role-players on his mock battlefield. And he expects, culturally, that they will remain Islamic.