Modern humans with non-African ancestry get between one and four percent of their DNA from Neanderthals. This species of hominid lived in Europe, the Middle East and central Asia until roughly 30,000 years ago, but their DNA has endured through the years—passed on to many populations of Homo Sapiens during a period of interbreeding between the species that began roughly 50,000 years ago.

So what happened to the rest of that Neanderthal DNA? Why don’t contemporary non-African humans have more than just a few percent of Neanderthal genes?

The reason likely has to do with population genetics, reports Steph Yin at The New York Times. Neanderthals lived in small, geographically isolated bands, which likely led to some negative consequences—genetically speaking.

“Neanderthals have this small population over hundreds of thousands of years, presumably because they’re living in very rough conditions,” Graham Coop, genetics professor at the University of California, Davis, and co-author of a new study published this week in the journal PLOS Genetics, tells Yin. So researchers used a mathematical model of natural selection to see how their population size influenced the number of genes they passed along. The results suggest that the small size of the Neanderthal population compared to the larger human population could account for why Neanderthal DNA isn’t a larger slice of the modern genome, Yin reports.

Over time that small population size meant Neanderthals began interbreeding with distant relatives, picking up negative genetic mutations, or "weakly deleterious gene variants," writes Eva Botkin-Kowacki at The Christian Science Monitor. But because the population was small, natural selection did not winnow these mutations out. Instead, they accumulated within the Neanderthal genome for thousands of years.

When Neanderthal’s began mating with humans, however, who had a much larger population, genetics did not favor keeping those genes, "purging" the mutations from their genomes, Ivan Juric, a geneticist at 23andme and co-author of the study tells Yin.

“The human population size has historically been much larger, and this is important since selection is more efficient at removing deleterious variants in large populations,” Juric says in a press release. “Weakly deleterious variants that could persist in Neanderthals could not persist in humans. We think that this simple explanation can account for the pattern of Neanderthal ancestry that we see today along the genome of modern humans.”

Coop says in the release that if Neanderthals had a larger population size that could naturally breed out the mutations or if humans weren’t quite so numerous it’s likely that non-African humans would carry much more Neanderthal genetic material.

The study jives with another paper published earlier this year computing the likelihood of negative mutations in Neanderthal and early human populations. “Our simulations showed that early hybrids would have been much less fit than pure humans,” Kelley Harris of Stanford, one of the paper’s authors, tells Yin. That would have meant much of their DNA would be left behind during natural selection.

In fact, Rasmus Nielsen, co-author of that study tells Botkin-Kowacki that he believes Neanderthals didn’t go extinct in the traditional sense. “They just simply got absorbed into the human species … Neanderthals were not driven to extinction by competition with humans—or by warfare with humans,” he says. “Rather they disappeared by simply slowly interbreeding with humans and by becoming part of the human species.”