A new genetic analysis of modern dogs and wolves suggests that man's best friend was domesticated before agriculture.

But the origin of this domestication remains stubbornly mysterious. Researchers analyzed the genomes of wolves from three likely sites of domestication (the Middle East, Asia and eastern Europe), and found that modern dogs were not more closely related to any of the three. In fact, it seems that the closest wolf ancestors of today's dogs may have gone extinct, leaving no wild descendants.

"The dogs all form one group, and the wolves all form one group, and there's no wolf that these dogs are more closely related to of the three that we sampled," said study researcher John Novembre, a professor of genetics at the University of Chicago. "That's the big surprise of the study." [10 Things You Didn't Know About Dogs]

Domestication mystery

The origin of the domestic dog is a persistent mystery. Fossil evidence for domestication dates back as far as 33,000 years, based on the shape of the skull and on ancient DNA analysis. But the presence of a dog-like canine doesn’t prove the origin of modern dogs; even if the fossil represents a domesticated dog, it could have been a failed lineage that left no descendants.

Researchers know that dogs regularly lived with humans by about 10,000 years ago, and dogs and people are found buried together as early as 14,000 years ago. Various genetic studies have pointed to China, the Middle East and Europe as the origin for today's domesticated dogs.

Novembre and his colleagues wanted to refine the understanding of domestication using high-quality, full genomes. They gathered full gene sequences from a wolf in Israel, a wolf in China and a wolf in Croatia to encompass the possible sites of the original dog domestication. Next, they also sequenced the full genomes of an Australian dingo, a feral dog species thought to have originated in Southeast Asia, and an African basenji. Neither of these dogs have territories that overlap with wolves, so researchers hoped they would see little of the post-domestication interbreeding that so often confuses the story of how dogs and wolves split.

The researchers also had a previously done full genome sequence for a Boxer.

Complicated canines

The high-quality, full sequences allowed the researchers to look at genetic variations across the entire genome. That's important, Novembre told LiveScience, because previous work was limited to snippets of DNA, chosen because they were known to vary from dog breed to dog breed. [The Coolest Animal Genomes]

"When we apply these to looking at dogs and wolves, we don't get a complete picture, because we can't see the variations that existed in wolves but vanished in dogs," Novembre said.

The new results, published today (Jan. 16) in the journal PLOS Genetics, reveal that dogs do not hail from the same lineage as modern wolves — a big surprise, said Novembre, who was hoping to see evidence for either a single domestication or multiple domestication events, where, for example, the Australian dingo would be most related to the Asian wolf and the African basenji would be most related to the Middle Eastern wolf.

Instead, the dogs are all most closely related to each other. The pattern suggests that dogs arose from a now-extinct line of wolves, Novembre said. Later, early in domesticated doggie history, they interbred with still-wild wolves, causing a genetic snarl that frustrates dog genetics researchers to this day.

The sequences also revealed that the first dogs arose from a very small number of the wolves that lived in their day, Novembre said. Around the time of domestication, both wolves and dogs experienced what's known as a population bottleneck — their numbers dropped. Genes can't explain why these drops occurred, Novembre said, but in the case of wolves, human encroachment and competition for large prey probably played a role.

Finally, the comparisons suggest that wolves and dogs split between 9,000 and 34,000 years ago, with a likely interval being between 11,000 and 16,000 years ago, before the rise of agriculture. Those findings are in line with the fossil record, Novembre said.

Previous research had suggested that perhaps dog domestication got a push from a genetic mutation that made it easier for modern dogs' ancestors to digest starch — meaning they could scavenge from human garbage piles. The new study looked at that gene mutation and found that it certainly occurred, but likely after dogs were already domesticated. Dingos, for example, are unquestionably dogs and not wolves, but they have few copies of the starch-friendly gene.

"You had domestication occurring in the context of dogs hanging around human hunter-gatherer groups, and only later, when these groups began to switch to farming, did they change their diets," Novembre said.

More answers coming?

However, there are still many questions left to answer. The reason for such a wide range of 25,000 years for the origin of domestication is that researchers had to base the estimate on rates of mutation in the genome. Mutations are rare, Novembre said, and estimating how often they happen is a tricky proposition. The best way is to compare the genomes of parents and offspring, but that work has not yet been done with dogs. Once it's done, Novembre said, the team will be able to refine its estimates.

However, the discovery that modern wolves and modern dogs seem to be more like sister groups than ancestors and descendants means that modern DNA sequences likely won't reveal the origin of domestication. To answer that question, Novembre said, ancient DNA analyses will be necessary.

So far, DNA sequences extracted from fossils are incomplete. But just as researchers have now sequenced a complete Neanderthal genome, they're on the cusp of sequencing full genomes from fossil dogs and wolves.

"Several groups are hammering away" at the problem, Novembre said, adding that a full ancient dog genome could be as few as nine months away.

Follow Stephanie Pappas on Twitter and Google+. Follow us @livescience, Facebook & Google+. Original article on LiveScience.

Copyright 2014 LiveScience, a TechMediaNetwork company. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.