On the ground floor of the Natural History Museum in London, arrays of Formica-covered cabinets stretch from floor to ceiling and from one end of the great building to the other. Some of nature's finest glories are stored here: pygmy hippo bones from Sicily, mammoth tusks from Siberia and skulls of giant sloths from South America.

Many treasures compete for attention, but there is one sample, kept in a small plywood box, that deserves especial interest: the Swanscombe skull. Found near Gravesend last century, it is made up of three pieces of the brain case of a 400,000-year-old female and is one of only half-a-dozen bits of skeleton that can be traced to men and women who lived in Britain before the end of the last ice age. Human remains do not get more precious than this.

However, the Swanscombe find is important for another, crucial reason: the skull is that of a Neanderthal, that race of shadowy, evolutionary cousins of our own species who made complex stone tools and who once thrived in Europe before being wiped about 35,000 years ago, not long after modern humans had emerged from their African birthplace and had begun to spread across the planet.

"This woman was clearly a member of a very successful tribe of

hunter-gatherers to judge from the thousands of stone axes they left

behind at Swanscombe," says Professor Chris Stringer, research leader

in human origins at the museum.

The reason for the Neanderthals' extinction has been pondered by scientists for 150 years without resolution. These bulky but brainy people have stubbornly refused to give up their secrets. However, a series of remarkable projects has recently shed new light on the Neanderthals and their relationship with modern humans. For the first time, scientists think we may unravel the mystery of their fate. It is a remarkable story that takes us from London to fossil sites across France, Spain, eastern Europe and Africa and – most important of all – to the Max Planck Institute for Evolutionary Anthropology in Leipzig.

The contrast between the institute and the dusty glories of the Natural History Museum could not be greater. The central concourse of this huge glass and concrete building has been fitted with a climbing wall – four storeys high – while a baby grand piano stands, mysteriously, at its foot. "We are encouraged to breakfast, lunch and dine here and to swap ideas," explains geneticist Svante Pääbo. "Hence the piano and climbing wall. We need to take our minds off things occasionally."

Certainly, its researchers deserve relaxation. Set up in 1997 and lavishly funded by the German government as part of its reunification of the country, the institute – in Leipzig, in the former German Democratic Republic – has already pioneered some striking research which culminated this year with completion of the sequencing of a Neanderthal genome.

By any account, this was an extraordinary undertaking. Scientists only succeeded in unravelling the three billion units of DNA that make up the human genome in 2000. Yet within 10 years, Pääbo did the same for a species that had died out more than 30,000 years ago using only bones from Vindija cave in Croatia as his source material. It was simply "fabulous" research, says Ian Tattersall of the American Museum of Natural History.

To create the genome, Pääbo and his team used dentists' drills to extract pill-sized samples of bone cells which were then broken up to reveal the DNA in their nuclei. The risk of researchers contaminating samples with their own DNA required them to wear full-body suits, masks and gloves all the time while air pressure in the laboratory was kept high so no contamination could blow in. And when the team went home, the room was irradiated.

It was a draining, four-year effort. Yet it produced results, with Pääbo outlining the 60% complete Neanderthal genome in May. "Now we can look for areas in the human genome where a change may have swept rapidly through us since we separated from Neanderthals," he said.

However, there was an added sensational aspect to Pääbo's work. He compared his Neanderthal genome with those of five modern humans, from South Africa, western Africa, France, China and New Guinea and found the last three possessed small amounts of Neanderthal DNA. The explanation is intriguing.

At some point during humanity's exodus from Africa, interbreeding between the two species must have taken place, probably when they met 100,000 years ago as Homo sapiens began to move into Neanderthal territory in the Levant. Then, as our species continued to spread round the globe, that Neanderthal DNA was carried with us. Hence it appears in non-Africans but not in Africans. The idea that humans and Neanderthals made love and not war millennia ago generated headlines round the world. "Our work is not about understanding Neanderthals," added Ed Green, one of Pääbo's team. "It's about understanding us."

Certainly, the notion that modern humans and Neanderthals were not just evolutionary cousins but were evolutionary kissing cousins does suggest differences between us may have been relatively slight. We had to be close to interbreed, surely? Thus it was just bad luck that doomed them and not us, a point stressed by Clive Finlayson, of the Gibraltar Museum.

He describes Neanderthals as expert ambush hunters who used thick vegetation to stalk their prey. By contrast, modern humans, who had evolved on the African savannah, were better long-distance runners who could chase their prey over open land. Crucially, as modern humans – or Cro-Magnons, as these early invaders of Europe are also known – entered Europe, climatic change began thinning its dense woodlands and opening up the landscape, giving them a key advantage over Neanderthals. "The roulette wheel of life favoured one and not the other in a particular place and time. A slight change of time, place, climate or fortunes and a Neanderthal might have been writing these lines," Finlayson wrote recently in the Times.

In short, we just got lucky. Apart from some differences in physique and behaviour, we were really very close to one another, it is argued. It sounds plausible. Yet many other scientists dispute this idea and point to a series of recent studies which undermine notions that Neanderthals were the intellectual equals of modern humans. A powerful example is provided by Tom Higham at the Oxford Radiocarbon Accelerator Unit who has used new dating techniques to study artefacts found at Grotte du Renne.

In the 1950s, archaeologists at this key site in central France dug up delicately carved pieces of bone and pierced teeth that had been worn as necklaces. These were attributed – by studying soil layers – to Neanderthal craftsmen and hailed as clear demonstrations of their artistic genius and capacity for symbolic expression. In fact, the artefacts are just about the only evidence so far uncovered which shows Neanderthals had artistic and symbolic talents. Hence the importance of Grotte du Renne.

Higham's work was published in the Proceedings of the US National Academy of Sciences last month and made uncomfortable reading for many researchers. He found that some pieces of Grotte du Renne jewellery that were attributed to Neanderthal "craftsmen" were only 21,000 years old – 10,000 years after the last Neanderthal had died. As timing issues go, this is a tricky one. As Higham says: "This is a site with problems."

In fact, it is now clear that layers of soil must have become mixed up over the millennia and that delicate artwork attributed to Neanderthals was the handiwork of Cro-Magnons. "Thus the single most impressive and widely cited pillar of evidence for the presence of complex symbolic behaviour among the Neanderthal populations in Europe has now effectively collapsed," says Paul Mellars of the University of Cambridge.

In short, the idea that Neanderthals possessed the same capacity for sophisticated expression and thought as modern humans has received a possibly mortal blow, raising serious questions about their intellectual prowess and ability to withstand their rivals, Homo sapiens. This is not to say that Neanderthals were grunting brutes, as was once alleged. They almost certainly possessed considerable skill, some language and a fair measure of compassion to judge from the number of Neanderthals who had clearly been cared for by their tribes after suffering injuries.

Nevertheless, it seems clear that an "intellectual and social gulf" separated them from Homo sapiens, says archaeologist Brian Fagan, in his new book, Cro-Magnon. "What gave our ancestors the edge was their intellectual awareness and imagination, their ability not only to co-operate with others but also to plan ahead and think of their surroundings as a living, vibrant world." We were a class apart, in other words.

But which specific traits gave us such an advantage that we were propelled to global glory at the expense of the Neanderthals? In the suite of behaviours that we evolved in Africa 150,000 years ago, what were the characteristics that really made a difference and can therefore be considered as defining human attributes? There are many candidates – complex language and superior memory, for example. However, among many scientists there appears to be consensus that imagination and opportunism were critical attributes.

This meant, says Fagan, that we learned to use local materials – antler, bone and ivory – in ways Neanderthals simply could not imagine. In one case, this resulted in "one of the most revolutionary inventions in history: the eyed needle, fashioned from a sliver of bone or ivory," he adds. While Neanderthals shivered in rags in winter, humans used vegetable fibres and needles – created by using stone awls – to make close-fitting, layered clothing and parkas: the survival of the snuggest, in short.

A slightly different interpretation is given by Steve Churchill of Duke University, in North Carolina. He believes it was the invention of the first projectile weapons that really did it for Homo sapiens. Bows and arrows as well as spear-throwers – special sticks with hooks – allowed hunters to hurl projectiles with greatly improved accuracy and distance. (An Aborigine still uses a spear-thrower known as a woomera.)

The consequences of these inventions were also profound. It meant Cro-Magnons could kill animals from a considerable – and rather safe – distance of between 20 and 40 metres. Neanderthals continued to fight close up and personal, often to their cost. "Spear-throwers meant you could fire a couple of shots off and not worry about being stamped on or trampled by an angry mammoth," says Churchill.

Evidence for this hypothesis is elusive, though Churchill has found that Cro-Magnon skeletons have signs of strain that suggest they were using such devices. Neanderthals do not.

However, it is unclear whether we used these weapons directly against Neanderthals. Most probably, we just got the best food long before they did. "If you look at other carnivore competitors living in Europe then – cave lions and cave hyenas – these also died out," added Churchill. "We took their food."

Stringer agrees: "If early modern humans had better ways of coping with rapid climate change, such as sewn clothing and more efficient weapons, they would have been more likely to survive. That is not to say modern humans always survived, since populations of both species only existed in their thousands in Europe then. Nevertheless, the effect would have been cumulative. We multiplied and the Neanderthals did not."