There are a couple of new papers out on how utterly fucked-up Neanderthals must have been, The Genetic Cost of Neanderthal Introgression by Kelley Harris and Rasmus Nielsen, and The Strength of Selection Against Neanderthal Introgression, by Ivan Juric, Simon Aeschbacher, and Graham Coop. The basic idea is the same in both: the effective population size of Neanderthals was considerably lower than that of anatomically modern humans (which makes sense, considering glaciation), and this had negative consequences. Theory says that selection is less effective in small populations: mutations with small negative effects (s < 1/2N) are not much noticed by selection and accumulate. Judging from the Neanderthal genomes we have sequenced, their effective (neutral) population size was about a tenth of modern humans. The first paper concludes that Neanderthals must have had much lower fitness than AMH: They say "Under an additive model, a recessive model, or anything in between, the severe reduction in fitness of Neanderthals would have doomed them to quick extinction if they had been competing for the same niche with humans under conditions of reproductive isolation. " Of course, one problem with this conclusion is that we know it is not correct. The projected Neanderthal excess genetic load would have accumulated over several hundred thousand years: it wouldn't have been much smaller 120,000 years ago (the Eemian) than it was ~50,000 years ago, when we displaced them. Modern humans in the Levant (Qafzeh/Shkul) ran into Neanderthals back in the Eemian – but they didn't win then : later Neanderthals seem to have reoccupied that area, and moderns don't seem to have developed the ability to displace Neanderthals until much later, some 50-60,000 years after their original contact with Neanderthals. Like they imply, if Neanderthals were that genetically screwed up, something like Florida panthers, we could have knocked them over with a feather – but it didn't happen. Presumably we would also see skeletal abnormalities, lots of them, in later Neanderthals – but I don't think we do.

Part of the problem is that the model is probably too simple. There is an argument, which makes sense to me, that suggest that small-N populations do better than you would think, because as the average population fitness gets farther from the optimum, strongly beneficial compensatory mutations become more and more possible. And the effective population size for generating those beneficial compensatory mutations is not the same as the neutral effective population size, which is dominated by the low points (probably glacial maxima) – it's much larger, closer to the average population size, instead of the harmonic mean.

In addition, I don't have much confidence in our models of the distribution of mutational effects, or our models of the way in which the effects of deleterious mutations add up.

Both papers talk about the likely genetic burden that Eurasians picked up from that Neanderthal admixture. Since East Asians have a somewhat higher level of Neanderthal admixture than people in Europe or the Middle East (~20% more) then they must have even more toxic Neanderthal genes, and Africans the least. This echoes earlier papers that have argued that population history (out-of-Africa bottleneck, Neanderthal admixture, etc) must have increased genetic load in Eurasians.

Evidently extra genetic load has anti-intuitive effects.