Little is known about the population history of Neandertals over the hundreds of thousands of years of their existence. We retrieved nuclear genomic sequences from two Neandertals, one from Hohlenstein-Stadel Cave in Germany and the other from Scladina Cave in Belgium, who lived around 120,000 years ago. Despite the deeply divergent mitochondrial lineage present in the former individual, both Neandertals are genetically closer to later Neandertals from Europe than to a roughly contemporaneous individual from Siberia. That the Hohlenstein-Stadel and Scladina individuals lived around the time of their most recent common ancestor with later Neandertals suggests that all later Neandertals trace at least part of their ancestry back to these early European Neandertals.

Without nuclear genome sequences from early European Neandertals, it has not been possible to determine the origin of the replacement and whether it was limited to the east. To learn more about the early population history of European Neandertals, we studied the nuclear genomes of two individuals from Western Europe that are dated to approximately 120 ka ago and from which only mitochondrial DNA (mtDNA) was previously recovered. The first, a femur from Hohlenstein-Stadel Cave (HST) in Germany ( 9 ), carries an mtDNA genome that falls basal to all other known Neandertal mtDNAs and was dated to ~124 ka ago based on its branch length in the mitochondrial tree [95% highest posterior density interval (HPDI), 62 to 183 ka ago; associated faunal remains suggest a date between 80 and 115 ka ago] ( 10 ). The second, a maxillary bone from Scladina Cave [Scladina I-4A, here referred to as Scladina ( 11 )], yielded the hypervariable region of the mtDNA genome ( 12 ) and was dated to 127 ka ago based on uranium and thorium isotopic ratios [1 SD, 95 to 173 ka ago ( 13 )].

Recent analyses of nuclear genome sequences from Neandertals have shown that, toward the end of their existence, Neandertals across their entire geographic range from Europe to Central Asia belonged to a single group sharing a most recent common ancestor less than 97 ka ago ( 6 , 7 ). However, population discontinuity has been observed in Denisova Cave, Russia, further back in time, where the Neandertal component in the genome of a ~90-ka-old Neandertal-Denisovan offspring ( 7 ) shows stronger affinities to late Neandertals in Europe than to the Altai Neandertal, another individual found in the same cave ( 8 ). The latter lived 120 ka ago according to the number of missing mutations in her genome compared to present-day human genomes. Thus, a population replacement likely occurred in the easternmost part of the Neandertal territory between 90 and 120 ka ago.

RESULTS

Because of the great age of the specimens and their extensive handling in the decades after their discovery, obtaining DNA of sufficient quantity for genome-wide analyses is challenging. We thus used the most efficient DNA extraction and library preparation methods currently available (14–16) and coupled them with pretreatment methods for the removal of human and microbial contamination (note S1) (17). We then characterized the libraries prepared from both specimens by hybridization capture of mtDNA and shallow shotgun sequencing to identify those libraries that were most suitable for further analysis (Materials and Methods; notes S2 and S3). On the basis of 450- and 107-fold coverage of the mtDNA genome, respectively, we were able to verify the published mtDNA sequence from HST (10) and reconstruct the complete mtDNA of Scladina (notes S5 and S6). Scladina dates to ~120 ka ago according to the branch length in the mtDNA tree (95% HPDI, 76 to 168 ka ago; note S7), consistent with the aforementioned date. Confirming previous results from the hypervariable region (10), we find that the complete Scladina mtDNA is most similar to the Altai Neandertal mtDNA (note S7). On the basis of only the mtDNA, it thus appears that both individuals fall outside the variation of later European Neandertals. However, mtDNA is only a single maternally inherited locus and of limited value for reconstructing the relationships among Neandertals and other archaic humans (1).

We generated a total of 168 and 78 million base pairs (Mbp) of nuclear DNA sequence from the two individuals, respectively (note S3). Ancient DNA sequences often carry cytosine to thymine substitutions that are caused by cytosine deamination accumulating in DNA fragments over time, most often at the ends of the fragments (18). The frequency of these substitutions on both molecule ends (1), confirms that ancient nuclear DNA is present but that a large proportion of the HST and Scladina sequences are contaminants from present-day humans (note S8). At positions that are derived only in the Altai Neandertal [ancestral in the genomes of a Denisovan (19) and an Mbuti (19)], 57.8 and 31.1% of HST and Scladina sequences, respectively, show the Neandertal allele (note S9). However, sequences also match the derived allele in an Mbuti genome (19) more often than the high-coverage genome of the Altai Neandertal does (HST, 8.8%; Scladina, 22.3%, Altai Neandertal, 1.4%; note S8). This excess of sharing suggests that 23 and 65% of the HST and Scladina sequences, respectively, are modern human contaminants (note S8). To reduce contamination, we restricted all further analyses to sequences that show evidence for deamination (Materials and Methods), leaving us with 51 Mbp of the HST genome and 12 Mbp of the Scladina genome (note S3). This procedure reduces the estimated contamination to 2% for HST and 5.5% for Scladina and results in a coverage on the X chromosome and autosomes that shows that HST was male, whereas Scladina was female, in agreement with the morphological assessments (notes S4 and S8) (9, 13).

To investigate the relationship of HST and Scladina to Neandertals, we compared their nuclear sequences to two high-coverage Neandertal genomes. The genome of a ~50-ka-old Neandertal from Vindija Cave in Croatia [Vindija 33.19, referred to as Vindija (20)] is a representative of the group of later Neandertals that inhabited Eurasia after 90 ka ago (6, 7), whereas the Altai Neandertal represents the earlier group of Neandertals in the east. We identified Vindija-specific– and Altai-specific–derived variants by randomly sampling an allele from these two Neandertal genomes and retaining only those variants that differ from the other high-coverage Neandertal genome and from the Denisovan (19), one Mbuti (19), and several ape outgroup genomes (Materials and Methods) (21–24). At these sites, HST shares Vindija-specific alleles more often than Altai-specific alleles (531 versus 466; two-sided binomial test, P = 0.043), while no significant difference was observed for Scladina (110 versus 106; P = 0.838; Fig. 2 and note S9). Since the number of DNA sequences with putative deamination-induced substitutions is small for Scladina, we repeated this analysis including all sequences and found that Scladina then shows more Vindija-specific alleles than Altai-specific alleles (Scladina, 443 versus 321; P < 10−4; HST, 1676 versus 1326; P < 10−9; note S9). This cannot be accounted for by contamination with present-day human DNA, since the proportion of Neandertal ancestry in present-day humans is, on average, smaller than 3% (note S9). Thus, these results indicate that both HST and Scladina are more closely related to Vindija than they are to the Altai Neandertal.

Fig. 2 Genetic relationship of HST and Scladina to Vindija 33.19 and the Altai Neandertal. The three possible tree topologies relating these Neandertals (H/S, HST or Scladina) are depicted in the middle. Mutations occurring on the internal branch (white points) produce an allelic configuration (A, ancestral; D, derived) that is informative of the underlying tree topology. Genome-wide counts of sites with the described configurations are presented on both sides (HST on left and Scladina on right) on the x axis. Lighter colors correspond to results using the alignments to the human reference hg19 (original) and to both hg19 and the Neandertalized reference (no reference bias). The darker points are corrected for present-day human DNA contamination assuming 2.0 and 5.5% contamination in the deamination-filtered fragments from HST and Scladina, respectively. The Vindija-like configuration (red) is the most supported topology after correcting for reference bias and contamination. The two other topologies are the result of incomplete lineage sorting and are equally likely. Bars represent 95% binomial CIs.

If HST and Scladina truly have a most recent common ancestor with Vindija more recently than with the Altai Neandertal, then their genomes are expected to share derived alleles with the Altai Neandertal genome as often as the Vindija genome does. However, the genomes of Vindija and the Altai Neandertal share more derived alleles with each other than the HST or Scladina genomes share with either of them (Fig. 2 and note S9). This imbalance in allele sharing can largely be accounted for by a reference bias that favors the alignment of HST and Scladina sequences that carry a modern human reference allele over those carrying a Neandertal allele (note S9). By aligning to an alternative reference genome containing alleles seen in the high-coverage Neandertals, we recover further sequences that we combine with the original set of alignments and compensate for this bias (Fig. 2, Materials and Methods, and note S9). The remaining imbalance in allele sharing can be explained by contamination and sequencing errors in Scladina and HST (Fig. 2 and note S9).

Using the reference bias–corrected alignments and two methods, we estimated split times between the populations represented by HST and Scladina and the Vindija population (note S10). Our first estimates are based on the sharing of derived alleles by HST/Scladina at sites where the high-coverage Vindija genome is heterozygous [F(A|B) statistic (8, 20)]. The estimated split times of HST and Scladina from the ancestor with Vindija are 101 ka ago [confidence interval (CI), 80 to 123 ka ago] and 100 ka ago (CI, 66 to 153 ka ago), respectively. The second estimates are based on a coalescent divergence model (25) and suggest, for both Neandertals, a ~10-ka-long shared history with Vindija after the split of the latter from the Altai Neandertal population (i.e., 122 to 141 ka ago, assuming 130 to 145 ka ago for the Vindija-Altai split time; note S10). The estimates from both methods are close to the estimated age of ~120 ka ago for these individuals (10, 13). Therefore, HST and Scladina could be members of an ancestral Neandertal population that gave rise to all Neandertals sequenced to date with the exception of the Altai Neandertal, who did not leave any descendants among sequenced Neandertals. This ancestral Neandertal population was established in the west by ~120 ka ago, and later descendants may have migrated east and replaced at least partially the eastern population of Neandertals represented by the Altai Neandertal.

It seems unexpected that HST carries an mtDNA lineage that diverged ~270 ka ago from other mtDNAs, given the recent population split times from the Vindija ancestors and the low levels of genetic diversity in the nuclear genomes of Neandertals (8, 20). To test whether such a deeply diverging mtDNA lineage could be maintained in the Neandertal population by chance, we used coalescent simulations with a demography estimated from the high-coverage Neandertal genomes (20), which was scaled to match the lower effective population size of the mtDNA, taking into account the difference in effective population size between the two sexes (8). We find that population split times between HST and other Neandertals of less than 150 ka ago make the occurrence of a mitochondrial time to the most recent common ancestor (TMRCA) of 270 ka ago unlikely (1.2% of all simulated loci have such a deep TMRCA; note S11). We note that this result is robust to uncertainties in the estimates of the Neandertal population size and of the mitochondrial TMRCA (note S11). The presence of this deeply divergent mtDNA in HST thus suggests a more complex scenario in which HST carries some ancestry from a genetically distant population.