Only a bit of the DNA coiled inside the cells of Europeans and Asians comes from Neandertals, but those snippets have sparked a flurry of research. In the past few years, researchers have traced them to one or two ancient encounters with our extinct cousins. Now, a report published online in Science this week details a far richer sexual past for modern humans and their archaic cousins, one that played out at multiple times and places over the past 60,000 years.

By developing powerful new statistical methods, an international team has identified how often and on which continents modern humans, Neandertals, and a second kind of archaic human called Denisovans met and mated. The researchers conclude that if you’re an East Asian, you have three Neandertals in your family tree; Europeans and South Asians have two, and Melanesians only one. (Africans, whose ancestors did not mate with Neandertals, have none.) Add in two additional liaisons known only from fossil DNA, and the ancestors of modern humans and Neandertals mixed it up at least five times. (Any matings that produced no offspring can’t be traced.) Meanwhile, the Denisovans bred at least once with Melanesians. “It was apparently separate events, so not just one single happy party at some point,” says evolutionary biologist Alan Cooper of the University of Adelaide in Australia, who was not part of the new study.

When researchers first spotted traces of Neandertal nuclear DNA in living people, they assumed that it must have come from a rare mating or two, likely when modern humans left Africa and first pushed into Neandertal territory in western Asia. But since then, the family history of modern humans and their cousins has grown tangled.

First, researchers found that Melanesians have inherited 2% to 4% of their DNA from the Denisovans, known from fossils at least 50,000 years old from Denisova Cave in Siberia, Russia. Then, they found that Denisovans had also interbred with Neandertals. Last year, researchers discovered that a 40,000-year-old modern human from Romania had a Neandertal great-great-great grandparent—but this Neandertal genome does not live on in present-day humans (Science, 22 May 2015, p. 847). Last month researchers reported modern DNA in a Neandertal toe bone, suggesting another early mating between the two types of humans, perhaps 100,000 years ago or so. And this week, the same team reports interbreeding between Denisovans and 440,000-year-old ancestors of Neandertals.

All this mixing has resulted in a patchwork of archaic DNA segments in modern human genomes. Tracing the source of each segment isn’t easy, in part because Neandertals and Denisovans were closely related. In Melanesians, for example, “up to 20% of the time when we say a segment matches a Neandertal, it’s Denisovan,” says population geneticist Joshua Akey of the University of Washington, Seattle, who led the new study.

So Akey’s team developed a statistical method to help identify and classify archaic DNA more reliably. They sequenced the genomes of 35 Island Melanesians, who carry more archaic DNA than any other group, and also analyzed the genomes of nearly 1500 other people around the world. They used the sequence data to find chunks of DNA likely inherited from archaic ancestors. Then they used new statistical methods to see how reliably they could classify the archaic DNA as Neandertal or Denisovan, and whether different populations had the same source of archaic DNA.

Akey expected that diverse Neandertal ancestors had contributed to Melanesian genomes, but that’s not what the team found: Most of the Melanesians’ archaic DNA turned out to be from Denisovans. What Neandertal DNA they have stems from a single liaison—the first one, soon after modern humans left Africa. A second Neandertal encounter shows up in the genomes of Europeans, South Asians, and East Asians, and likely happened in the Middle East before these populations diverged. Finally, the ancestors of East Asians had a third hookup with Neandertals, presumably somewhere in Asia (see graphic, below).

The most likely explanation, Akey says, is that Melanesians split from the ancestors of Europeans and Asians before the second encounter. Later, East Asians broke away from Europeans and South Asians and got a third pulse of Neandertal DNA as they went their separate way. Meanwhile, the ancestors of Melanesians picked up genes from Denisovans somewhere in Asia. The legacy of this ancient sex includes 21 chunks of archaic DNA bearing immune genes that recognize viruses, along with several metabolism genes, such as GCG, which increases blood glucose levels, and PLPP1, a cell membrane protein that breaks down fats. These genes may have helped modern humans adapt to new diseases, diets, and climates as they moved into Neandertal territory in Europe and Asia.

But the researchers also found “deserts”—stretches of the human genome where no archaic DNA appears. These genomic regions may once have carried the legacy of ancient encounters, but it no longer survives in living people. That suggests that these parts of the genome, which contain genes linked to language, brain development, and autism, are critical to a modern human’s identity and reproductive fitness: Archaic gene variants can’t be tolerated here.

Some researchers caution that modern DNA may not be a completely reliable guide to ancient matings. Groups like Melanesians may have simply lost archaic genes over time, so that their full mating history with Neandertals was erased. “It seems possible that the differences in Neandertal ancestry between present-day people could be due to differences in the efficacy of natural selection ‘weeding out’ Neandertal segments,” says population geneticist Pontus Skoglund of Harvard University.

Still, he and others praise the power of the new method and the pace at which researchers are building a catalog of DNA inherited from archaic humans, which keeps adding new twists to the story of our origins. As evolutionary biologist Eske Willerslev of the University of Copenhagen and the University of Cambridge in the United Kingdom notes: “It seems like [ancient admixture] keeps getting more complicated.”