Image caption Did Palaeolithic hunters leave a genetic legacy in today's European males?

A new study deals a blow to the idea that most European men are descended from farmers who migrated from the Near East 5,000-10,000 years ago.

The findings challenge previous research showing that the genetic signature of the farmers displaced that of Europe's indigenous hunters.

The latest research leans towards the idea that most of Europe's males trace a line of descent to stone-age hunters.

But the authors say more work is needed to answer this question.

The study, by an international team, is published in the journal Proceedings of the Royal Society B.

I would say that we are putting the ball back in the middle of the field Dr Cristian Capelli, University of Oxford

Archaeological finds show that modern humans first settled in Europe from about 40,000 years ago - during a time known as the Palaeolithic.

These people survived an Ice Age some 20,000 years ago by retreating to relatively warm refuges in the south of the continent, before expanding into northern Europe again when the ice melted.

But just a few thousand years after Europe had been resettled by these hunter-gatherers, the continent underwent momentous cultural change. Farmers spread westwards from the area that is now Turkey, bringing with them a new economy and way of life.

The extent to which modern Europeans are descended from these early farmers versus the indigenous hunter-gatherers who settled the continent thousands of years previously is a matter of heated debate.

The results vary depending on the genetic markers studied and are subject to differing interpretations.

Family tree

The latest study focused on the Y chromosome - a package of DNA which is passed down more or less unchanged from father to son.

The Y chromosomes carried by people today can be classified into different types, or lineages, which - to some extent - reflect their geographical origins.

More than 100 million European men carry a type called R-M269, so identifying when this genetic group spread out is vital to understanding the peopling of Europe.

R-M269 is most common in western Europe, reaching frequencies of 90% or more in Spain, Ireland and Wales.

Image caption The Neolithic was a time of momentous cultural change in Europe

But while this type reaches its highest distribution on the Atlantic fringe, Patricia Balaresque and colleagues at the University of Leicester published a paper in 2010 showing that the genetic diversity of R-M269 increases as one moves east - reaching a peak in Anatolia (modern Turkey).

Genetic diversity is used as a measure of age; lineages that have been around for a long time accumulate more diversity. So this principle can be used to estimate the age of a population.

When the Leicester team estimated how old R-M269 was in different populations across Europe, they found the age ranges were more compatible with an expansion in Neolithic times (between 5,000 and 10,000 years ago).

The team's conclusions received support from papers published in August 2010 and in June this year. But one study which appeared last year backed the idea of a more ancient, Palaeolithic origin for R-M269.

Age estimates

Now, a team including Cristian Capelli and George Busby at Oxford University have explored the question.

Their results, based on a sample of more than 4,500 men from Europe and western Asia, showed no geographical trends in the diversity of R-M269. Such trends would be expected if the lineage had expanded from Anatolia with Neolithic farmers.

Furthermore, they suggest that some of the markers on the Y chromosome are less reliable than others for estimating the ages of genetic lineages. On these grounds, they argue that current analytical tools are unsuitable for dating the expansion of R-M269.

Image caption Studies of DNA from ancient remains could shed more light on European origins

Indeed, Dr Capelli and his team say the problem extends to other studies of Y-chromosome lineages: dates based on the analysis of conventional DNA markers may have been "systematically underestimated", they write in Proceedings B.

But Dr Capelli stressed that his study could not answer the question of when the ubiquitous R-M269 expanded in Europe, although his lab is carrying out more work on the subject.

"At the moment it's not possible to claim anything about the age of this lineage," he told BBC News, "I would say that we are putting the ball back in the middle of the field."

Co-author Dr Jim Wilson from the University of Edinburgh explained: "Estimating a date at which an ancestral lineage originated is an interesting application of genetics, but unfortunately it is beset with difficulties."

The increasing frequency of R-M269 towards western Europe had long been seen by some researchers as an indication that Palaeolithic European genes survived in this region - alongside other clues.

A more recent origin for R-M269 than the Neolithic is also possible. But researchers point out that after the advent of agriculture, populations in Europe exploded, meaning that it would have been more difficult for incoming migrants to displace local people.

Paul.Rincon-INTERNET@bbc.co.uk