The movie Blood Diamond opens with a ferocious raid by rebels on a quiet village in Sierra Leone. Families are scattered and separated as the rebels rampage, murdering indiscriminately and setting fire to thatched-roofed huts and smallholdings. When the killing stops, some of the stronger men from the village who have survived are rounded up to work as slaves in the rebel-controlled diamond mines in the east of the country and their sons are taken away in trucks, destined to become boy soldiers in one of Africa's harshest civil wars.

Blood Diamond is a complex film. On one level it indulges many of the stereotypes of the traditional Hollywood adventure movie set in or about Africa. For a start, the hero, a Rhodesian mercenary, played by an impressive Leonardo DiCaprio, is white. The chaos of a war-stricken African state provides the setting for his quest to live authentically and free, apart from conventional bourgeois society. He is an all-action hero, ruthless in his pursuit of self-affirmation and seemingly indestructible until, that is, his heart is softened by a woman - an American journalist, who is, also, white.

Yet on another level, Blood Diamond is an important political text which, as did The Constant Gardener before it, attempts to expose the complicity of western corporations in the ruin of a resources-rich but vulnerable and disturbed African country; the ultimate villains, for all their cruelty and violence, are not the rebel soldiers but their western sponsors. Most interesting, the film investigates the phenomenon of the child soldier in Africa, of those boys and sometimes girls who are orphaned in conflict or stolen from their families and then brutalised and humiliated until, finding a new kind of family among rebel soldiers, they become drug-addicted killers, without pity or fear.

We know something of these children from innumerable journalistic dispatches from northern Uganda, where the paramilitary Lord's Resistance Army has fought a murderous war against the Museveni government for nearly two decades; from Sierra Leone, whose civil war was ended by one of Tony Blair's more successful foreign policy interventions; from Liberia, one of the most ravaged of all African states; and, most recently, from Sudan, where the attacks on villages by Arab militias, mobile killing units inspired by the Khartoum government, continue unhindered in spite of the presence of African Union peacekeeping forces and agitation from western aid groups such as the admirable Aegis Trust.

We know something of these children, too, from recent novels, such as Uzodinma Iweala's acclaimed Beasts of No Nation, which was set in an unnamed west African state and narrated in a swirling, fractured demotic by a boy soldier named Agu, and Ahmadou Kourouma's Allah is Not Obliged, which takes as its setting the civil war in Liberia, as well as from Blood Diamond.

We know something of these children but, at the same time, can we ever say that we know them - know what they feel, think, need or want in all their complicated interiority? The honest answer is that we do not, although that may be about to change with the publication next month of two remarkable first-person documentary accounts. These are Ishmael Beah's A Long Way Gone, an astonishingly self-revealing memoir of his time as a boy soldier in Sierra Leone in the mid-1990s, and Dave Eggers's What Is the What, a long, experimental non-fiction novel written in collaboration with Valentino Achak Deng, one of the so-called Lost Boys of Sudan who are now living in the US, having been displaced in the civil war. There is, too, the forthcoming documentary film God Grew Tired of Us, directed by Christopher Quinn, with Brad Pitt as an executive producer and Nicole Kidman as narrator. Like the Eggers novel, this is a story of American immigration framed by the trauma of war in Sudan and the suffering of tens of thousands of children separated from their families, some of whom became boy soldiers.

Clearly, as the New York Times recently put it, the African boy-killer, or the 'kid-at-arms', in the American vernacular, 'is becoming a pop-cultural trope. He's in novels, movies, magazines and on TV, flaunting his Uzi like a giant foam hand at a baseball game'. His ubiquity is a manifestation of how far Africa, with its wars, post-colonial dilemmas and ethnic rivalries but also its hope, vibrancy, cultural and linguistic diversity, has moved from the margins to the centre of western consciousness; of how Africa has become the cause celeb, as it were, of our new globalisation.

It is obvious that cultural extravaganzas such as Africa 05, the Make Poverty History campaign and Live 8 have played their part in raising greater awareness of Africa, its problems and its needs, as has the proselytising of Bono, Bill Gates and Tony Blair. The critical and commercial success of works such as Blood Diamond and The Last King of Scotland, about Idi Amin's Uganda, have also reaffirmed what many executives in Hollywood no doubt suspected: African stories, especially African war stories, sell. They are the cultural news.

Even Starbucks, purveyors of grotesquely oversized corporate cappuccinos, has sensed a commercial opportunity. The coffee chain has chosen Beah's memoir as the inaugural title of its new book club. A Long Way Gone is about to go on sale in all Starbucks in Britain; in the US, it is already a number one bestseller because of the support of Starbucks, which sponsored Beah's countrywide reading tour.

As harrowing as it is violent, A Long Way Gone is an odd choice for a sanitised coffee chain. It begins relatively serenely, with a gentle portrayal of communal life in Beah's home village. One day the village is attacked by the Revolutionary United Front, the rebel army that controls Sierra Leone's diamond mines. Beah, then 12, is separated from his family in the ensuing chaos. Once safe, he first allows himself to believe that a reunion with his family is possible, supported and protected as he is by other displaced boys - in a kind of union of strangers. Later, however, he discovers that his parents and two brothers are dead; soon, he is separated from the boys as well, left to wander alone and bereft in the hostile bush, scavenging for food. He is eventually picked up by government troops in the south-east. They teach him how to handle an AK-47 in preparation for his becoming part of a child unit fighting the rebels in the bush.

Beah writes without self-pity of how he became addicted to drugs, 'smoking marijuana and sniffing brown brown, cocaine mixed with gunpowder', of how he ransacked villages, murdering innocents as and when he had to; and of how he learned never to wonder about the old life he left behind. The first death is the most difficult for him, after which he kills and tortures without remorse.

Beah has no idea how many people he killed. 'I never thought to keep count,' he told Time magazine. 'We attacked civilians, villagers, anyone the commander deemed was an enemy; we killed them. If you thought they maybe aided the rebels, you shot them. If they withheld food, you shot them.'

In 1998, he moved to New York to live with a woman called Laura Simms, who worked at the United Nations and had been paying his school fees in Freetown as part of his rehabilitation. He was 17. He attended Oberlin College in Ohio and decided to write a memoir because he knows he has been given a second chance in life, 'a second life even', and he wants to bear witness so that others may learn from his experience. He is tormented by guilty memories, but refuses to give in to them. 'If I choose to feel guilty for what I have done, I will want to be dead myself,' he says.

He writes with taut, pared-down urgency; when he tries to inflate his style, the effect can be awkward: 'Sometimes I closed my eyes hard to avoid thinking, but the eye of my mind refused to be closed and continued to plague me with images ...'

At times, too much is described too hastily, so that one narrow escape from death in the bush or raid on rebel positions reads very much like another, rendered in the hectic style of a pulp thriller. Except that, the story being told here is not a work of overheated imagination: it is painfully, terrifyingly true.

The publisher Fourth Estate is selling A Long Way Gone hard, perhaps overselling it: 'This account is utterly unique - until now there has not been a first-person account of this kind.' In fact, I read something similar only a couple of years ago, China Keitetsi's Child Soldier, published by the independent Souvenir Press. Keitetsi fought as a young girl in Yoweri Museveni's National Resistance Army, which came to power in Uganda in 1985. (Paul Kagame, now President of Rwanda, fought alongside Museveni, as did many Uganda-exiled Tutsis who later formed the Rwandan Patriotic Front and, in 1994, led the liberation of their home country following the genocide in which over a period of 100 days nearly one million people, mostly ethnic Tutsis, were murdered.)

Keitetsi, who lives in Denmark, was repeatedly raped as a soldier and had her first child at the age of 14. 'For us female soldiers, we had to offer sex to more than five officers in one unit,' she writes. 'Nearly every evening an officer would come and order you to report to his place, typically at 9pm ... I have never been to hell, though I am not always sure. Where else can such pain belong? It was so painful, but I could only cry with my heart, because with tears I could never survive.'

Uzodinma Iweala has just been selected as one of Granta magazine's Best of Young American Writers. 'It's important and wonderful that issues such as that of the plight of the child soldiers are being written about and discussed,' he says of books by Keitetsi, Beah and others. 'If people are encouraged as a result to think more deeply about the historical context for conflict, about the situation about which they are reading as well as the historical context of African writing itself, then that's all the better.'

There is, however, a problem, what he calls a 'sinister side' to our fascination with Africa. 'Is this resurgence of interest in Africa and its issues indicative of a genuine desire to see change?' he asks. 'Or does it merely reflect that Africa has been centre stage for the past two or three years, even if it is now being edged out by the new celebrity cause of global warming?'

Iweala is a Nigerian, an Igbo, born and educated in America. His mother, Ngozi Okonjo-Iweala, was until recently the finance and then foreign minister of Nigeria. Before that, she was vice-president of the World Bank and returned to Nigeria only on a mission to reduce corruption and introduce economic reforms. Her son is a child of privilege and opportunity, with an Ivy League education. So why for his first novel did he choose to speak in such a disturbed voice, as a child of suffering?

'The sun is dropping down behind the hill like it is not wanting to be seeing us anymore,' says Agu, the warrior-boy narrator of Beasts of No Nation. 'All we are knowing is that before the war we are children and now we are not.'

In other words, why war? Why boy soldiers?

Stuart Brown, of the Centre of West Africa Studies at the University of Birmingham, has spoken of how so much African writing is 'very serious' precisely because it has a sense of social responsibility. 'They [African writers] don't see themselves as entertainers in the same sense that American writers do. African writers feel it is their duty to represent their experiences in their world.'

But Iweala's experience could not have been more different from those he describes in his novel. Is he, in one sense, then, writing for a western audience, conforming to what is expected of a novel set in Africa? He is silent for a while and then says: 'As African writers we have a compulsion to tell our own stories. I agree that there can be an expectation for stories about Africa to be about suffering, death and destruction ... but for too long we had to listen to other people, non-Africans, as they attempted to tell our stories.'

The day after we had spoken, I received an email from Iweala. He had been thinking about what he called the West's sinister fascination with Africa and why, for all the comfort of his American lifestyle, he had written a novel of such violence, linguistic and thematic.

'If you read articles and watch the BBC you constantly and consistently hear words like 'bloodthirsty', 'tribal'. And the wars we fight are always 'senseless' as compared to western wars which I suppose are full of meaning. Personally I believe all war is senseless. What I'm saying is that I wonder if sometimes the fascination with the child soldier or Africa's wars is one of genuine 'how can we stop this' or whether it's that same attitude that people have when they pass a gruesome road accident.

'Now if that's what I think then why did I write about the topic? When dealing with a place you have to tell good stories and stories that are not necessarily so pleasant. Why I and others like me who have written and will write about such a subject is to humanise rather than dehumanise as the press and common stereotypes often do. In my book - and in Beah's book, and the others that have come before - there is much more focus on the internal state of the child soldier. We are attempting to understand why any human would be put in this situation or how any human would behave in this situation. I and I think others who write about the subject are committed to doing that through our work. Interestingly, when I started researching for my novel, I was reading about child soldiers around the world not just in Africa. In the former Yugoslavia, Chechnya, South Asia ...'

The most celebrated of the current batch of boy soldier narratives is, perhaps inevitably, What Is the What by the bestselling memoirist, novelist and literary entrepreneur Dave Eggers. Iweala is one of several of the new generation of African novelists to have endorsed the Eggers book - published here at the end of next month - with a cover quote.

What Is the What is a curious hybrid. It is at once a gripping, fast-paced adventure story - there are gun battles, people are eaten by crocodiles, and so on - and a more ruminative, coming-of-age narrative. It is a biography as well as a work of fiction, with invented scenes and characters. It is a book about America as much as it's a book about Africa - the novel begins with Deng being attacked and tied up in his Atlanta apartment by African-American raiders, who mock his accent and call him, pejoratively, 'Africa'.

Eggers has spoken of how he wanted to write a conventional biography, to tell Achak Deng's story straight. 'I didn't want my voice in there.' In the end, he opted, perhaps because of the difficulty of his self-imposed task, to become a kind of ventriloquist. He inhabits a role, and presumes to speak through Deng as if he were him. He need not have worried about indulging too much in an act of self-display, because his voice isn't really in there at all; the voice in which the fictional Deng tells his story has a flat, parable-like simplicity, simultaneously monotonous, innocent and beguiling. It's the voice of a hundred fairytales. And he speaks in the same measured tone throughout what is a 500-page novel, a formidable feat of consistency and control.

'In many cases, the Lost Boys of Sudan have no one else,' the fictional Achak tells us, beginning another of his long internal monologues. 'The Lost Boys is not a nickname appreciated by many among our ranks, but it is apt enough. We fled or were sent from our homes, many of us orphaned, and thousands of us wandered through deserts and forests for what seemed like years. In many ways we are alone and in most cases we are unsure of where exactly we're going.'

As a boy - aged no more than about six - Valentino Achak Deng was separated from his family when his village of Marial Bai in southern Sudan came under attack from Islamist Arab militia on horseback, the murahaleen. That was during what is now known as the second Sudanese civil war, from 1983 to 2005. (As for what is happening now in Darfur, western Sudan - the violence, the expulsions, the mass killing - if it is not a continuation of the same civil war, it is certainly genocide.)

Once adrift from his family, Achak, an ethnic Dinka, begins to wander lost and hungry with other similarly bereaved boys. Together in ever-increasing numbers they begin the monumental trek north to Ethiopia, where they have been told they will find safety and comfort, 'in the place that is', as well as one day having the possibility of being reunited with their families, if their families are not dead already.

Deng, who is now 25 and a student at Allegheny College in Pennsylvania, eventually arrived in America in 2001, having spent a decade at the Kakuma camp in Kenya, what he describes as 'one of the largest and most remote refugee camps in the world'. He was brought to America by a charity called the Lost Boys Foundation and once there he was introduced to Eggers, to whom, as he writes in the preface to What Is the What, he told his story orally over the course of many years. Eggers then 'concocted this novel, approximating my voice and using the basic events of my life as the foundation'.

Why didn't Achak Deng write his own story, as Ishmael Beah has? If he didn't have the confidence to write it himself, as Beah did, he could have hired an experienced ghost writer to do it for him. But perhaps such questions are glib; perhaps he was simply too caught up in the knots and entanglements of his own story. Perhaps only a writer with the patience, empathy and artistry of Eggers could have fully unravelled it for him. After all, Eggers himself is a kind of lost boy; his bestselling memoir, A Heartbreaking Work of Staggering Genius, tells of how the death of his parents while he was at college left him and his sister alone, with the responsibility of caring for their seven-year-old brother.

Deng has granted Eggers an unusual freedom, the freedom to recast his life in fiction. The result is a deeply affecting, if problematic work. Its chief selling point and the source of much of its considerable pathos is the claim it makes to truth - to be telling the truth, in broad outline, of the terrors of the civil war in Sudan as well as the truth, in detailed particularity, of one man's suffering and quest for redemption in America.

There is a moment, early in A Long Way Gone, when Ishmael Beah, in flight from rebel attack, describes arriving in a village in the company of a group of boys. This is before he has become a soldier, before his fall into annihilation and despair. Beah loves American rap music and carries cassettes of his favourite artists with him at all times. But on arriving in the village he is searched and his rap cassettes are confiscated. The village chief demands to know what kind of music is rap. 'It is similar to telling parables,' Beah replies, 'but in the white man's language.'

In their own different ways, Eggers and Deng, Beah and Iweala, and even China Keitetsi (whose book was published first in Danish before being translated into many languages), are all telling parables, parables of loss, flight and renewal, not in the white man's language, but in a style and idiom that is entirely their own. Absent from their books are the grand, melodramatic generalisations about Africa that are such a feature of so much work by western writers about the continent - written in what might be called the standard white man's language. Here, for instance, is Peter Godwin, from his recent memoir about Zimbabwe, When a Crocodile Eats the Sun: 'Most of us struggle in life to maintain the illusion of control, but in Africa that illusion is almost impossible to maintain. I always have the sense there that there is no equilibrium, that everything teeters on the brink of some dramatic change [in Africa], that society constantly stands poised for some spasm, some tsunami in which you can do nothing but hope to bob up to the surface and not be sucked out into a dark and hungry sea.'

The stories of Ishmael Beah, Achak Deng and the Lost Boys featured in God Grew Tired of Us are each related in long retrospective, from positions of relative safety in America. They are stories of departure and of arrival - in a new land, with new aspirations. They are also about the process of memory, of what and how we choose to remember and what we would rather forget, if only we could. When Beah describes his most brutal experiences as a soldier in the bush his language inevitably contracts; it reduces, becomes perfunctory. As the novelist William Boyd has smartly observed: 'The horror is duly registered, but its vagueness and generality don't add up to moments of lived personal history.'

It is as if such moments of horror can only be described in the sketchiest of details; to recall them otherwise would be beyond perhaps what is possible. It would be, as it were, to relive them all over again. 'Forgiveness is actually an important part of healing from the war for me,' Beah says. 'To forgive is not to forget but to transform all that happened into something positive because the other route can only bring more suffering to me and those around me.'

You hope that writing his book has brought him a release from suffering. As for the other countless lost boys and kids-at-arms who have survived wars but have never made it out of Africa, you would like to believe that one day they, too, will be able to find a kind of peace.

Child soldiers: the facts

Human Rights Watch estimates there are up to 300,000 child (under-18) soldiers worldwide, located in 36 countries, including Iraq, Colombia, Ethiopia and Russia.

Numbers of child soldiers have risen as small, easy-to-use automatic guns have become more widely available.

In some countries, such as Nepal, Sri Lanka and Uganda, more than a third of child soldiers are girls.

Many are addicted to hard drugs, which they are given to make them more courageous in battle.

They are often forced to commit atrocities against their family so that they can't return home.

A 2002 UN optional protocol raised the minimum age for recruiting soldiers from 15 to 18. The UK government is among 110 countries to have ratified it, but says it would still use under-18s if there was 'a genuine military need'.

Hugh Montgomery

· A Long Way Gone by Ishmael Beah is published by Fourth Estate, £14.99. To order a copy for £13.99, including UK p&p, go to observer.co.uk/bookshop or call 0870 836 0885. What Is the What by Dave Eggers is published in June by Hamish Hamilton, £18.99