The Early Pleistocene Homo individuals, which were projected onto the PC spaces using the above variance-covariance matrices, occupy a space between H. floresiensis and H. sapiens in the analysis of the mandibular teeth, suggesting that the pattern observed in H. floresiensis involves primitive morphology for the genus Homo. Such a trend is, however, not evident in the analysis of the maxillary teeth. In both analyses, the early Javanese H. erectus individuals (Sangiran 4 for the maxillary teeth and Sangiran 22 for the mandibular teeth) are comparatively close to the positions of H. floresiensis.

This unique tooth size proportion was also confirmed by the following multivariate analyses. Fig 3 and Table 1 are the results of principal component analyses based on size-adjusted MD and BL data. The generated PC1 and PC2 cumulatively explain 75% (maxilla) or 91% (mandible) of the total variation. These PCs show no (PC1 and PC2 for the maxillary analysis, and PC1 for the mandibular analysis) or only slight (PC2 for the mandibular analysis) correlations with the ‘crown size factor’ ( Table 1 ), and thus reflect crown shape variations that are mostly independent from crown size. In both analyses, the PC scores of the H. floresiensis teeth occupy unique positions relative to the H. sapiens individuals.

Z-scores are relative deviations from the H. sapiens means in units of standard deviation. Note that the Dmanisi Homo sample here is based on the two smaller individuals. Due to sever tooth wear the largest individual (D4500/2600) l was excluded [ 27 ].

We first analyzed tooth size based on crown length (MD) and breadth (LL or BL) data ( Fig 2 ). Many of the H. floresiensis teeth are within the smaller range of variation exhibited by the global H. sapiens sample. Remarkable deviations from this general trend are disproportionately long P 3 s as well as short M 1 and M 1 in H. floresiensis.

The positive and negative are reversed from Figs 4 , 5 and 6 in PC3 for P 4 , PC1 for M 1 , and PC1 for M 1 , for the sake of unanimity in the directions of variation. See Figs 4 , 5 and 6 for component loading for each PC. The shape variation reflected by each PC is shown in the upper row. Blue and red lines indicate contours of +2 SD and −2 SD of the entire sample, respectively. The green line is the grand mean. In all of these six cases, the contours of H. floresiensis are close to the red outlines, and those of H. habilis to blue outlines. Box plots of the PC scores are shown in the lower row. Note that H. habilis plots far from H. floresiensis whereas early Javanese H. erectus is closer to H. floresiensis in all these PCs.

When the PC score for the early Javanese H. erectus and the H. habilis samples are compared to each other, the former differs significantly from the latter in having MD short and BL wide M 1 (PC1) and M 2 (PC1) [ 62 , 71 ], and a BL symmetric M 2 (PC2) [ 72 ] (P < 0.05, Mann-Whitney U Test with Bonferroni correction). A non-triangular P 4 (PC3) [ 52 ], and a MD short (PC1) and BL symmetric (PC2) M 1 are also added to the above list if Bonferroni corrections are not made. In all of these six distinguishing characters ( Fig 8 ), the Liang Bua Pleistocene teeth are similar (P 4 , M 2 and M 2 ) or closer (M 1 and M 1 ) to early Javanese H. erectus but are remote from H. habilis ( Fig 9 ). Thus, the postcanine crown contours of H. floresiensis are derived relative to H. habilis and more similar to early Javanese H. erectus in many aspects.

Because the two chronological samples of early Javanese H. erectus (Sangiran Lower and Sangiran Upper) are essentially similar to each other in all the PCs, they were pooled for the following statistical analyses. In the PC scores plotted in Figs 4 , 5 and 6 , the pooled Early Pleistocene fossil Homo sample (H. habilis, H. ergaster, and early Javanese H. erectus) differ significantly from H. sapiens in eight out of the twenty-four PCs generated from the six EFAs (PC1 and PC2 for P 3 , PC3 for M 1 , PC1 and PC4 for M 2 , PC1 and PC3 for M 2 : P < 0.05, Mann-Whitney U Test with Bonferroni correction). In particular, PC1 and PC2 for P 3 in combination separate the modern and the Early Pleistocene samples nearly completely. Two other PCs also differ significantly if Bonferroni correction is not made (PC2 for P 4 , PC3 for M 1 ). Thus, these eight (with Bonferroni correction) or ten (without Bonferroni correction) PCs reflect primitive features for the genus Homo. Figs 4 , 5 and 6 shows that H. floresiensis shares all of these primitive features except for PC1 of M 1 . In PC 1 of M 1 as well as PC1 and PC2 for M 1 , H. floresiensis is distinct from both modern and fossil Homo (Figs 5A and 6A ). These primarily reflect the short MD diameters of the first molars ( Fig 2 ). In the H. sapiens sample, there is no evidence that smaller first molars approach the short configuration similar to H. floresiensis ( Fig 7B and 7C ). This further highlights the uniqueness of the latter.

(A) and (B) Mandibular first premolar. (C) and (D) Mandibular second premolar. Symbol and color codes: gray symbols = H. sapiens (crosses = Southeast Asia/Melanesia/Australia, circles = African Pygmy); colored letters = fossil Homo (L 1 = LB1, L 2 = LB2/2, L 6 = LB6/1, L 15 = LB15/1, S = early Javanese H. erectus (Sangiran Lower), s = early Javanese H. erectus (Sangiran Upper), D = Dmanisi, e = H. ergaster, h = H. habilis). The right and left teeth are included for H. floresiensis when available and they are indicated by the dashed line with arrow heads. The crown outlines for -2 SD, 0, and +2 SD, 95% confidence ellipses for the H. sapiens sample, and ranges for H. erectus and H. habilis samples are shown. Proportion of the variance explained by each PC is in the parentheses.

Because wear obscures much of the occlusal surface morphology of the H. floresiensis teeth, we analyzed the occlusal crown contours of six teeth (P 3 , P 4 , M 1 , M 2 , M 1 , and M 2 ) by normalized (size-adjusted) Elliptic Fourier Analysis (EFA) combined with PCA. In consideration of the previous claim that the Liang Bua Pleistocene hominins resemble a local short-statured Australo-Melanesian population [ 23 ], the H. sapiens sample used here includes prehistoric people with Australo-Melanesian affinities sampled from Flores, Java, Malaysia, and Vietnam [ 70 ], as well as Aboriginal Australians, Papuans, Philippine Negritos, and African Pygmies (N = 54 [P 3 ], 41 [P 4 ], 106 [M 1 ], 112 [M 2 ], 71 [M 1 ], 105 [M 2 ]: S2 Table ). Fossil Homo samples from Africa (H. habilis and H. ergaster) and Java (early H. erectus) were included ( S1 Table ). Within the H. sapiens sample, each PC1 shows a weak correlation with measured crown area (Pearson’s correlation coefficients were -0.17 [P 3 ], 0.22 [P 4 ], -0.13 [M 1 ], -0.26 [M 2 ], 0.15 [M 1 ], and 0.11 [M 2 ]; M 2 was the only tooth that reached statistical significance at the α level of 0.05), indicating that the size-adjustment was effective for these analyses. In all six analyses the first two PCs (PC1 and PC2) cumulatively explain 71−85%, and the next two (PC3 and PC4) 7−15% of the total variations (see Figs 4 , 5 and 6 for the value for each PC). These four PCs are considered below.

Other morphological traits

Twenty morphological characters of individual teeth (nos. 1−20) were assessed based on presence/absence with metric criteria when applicable, and three characters of the dentition (nos. 21−23) were evaluated based on linear metric data. Among these 23 traits listed in S3 Table, 16 were found to be of some use to evaluate dental morphological status of H. floresiensis. These 16 characters are reported in Table 2 and described below. The rest of the results are available in S3 Table and S1 Text. Differences in the frequency data were tested between the H. habilis and other samples, and between the H. sapiens and other samples (Table 2). Because the results showed no differences between the crown contours of the older and younger early Javanese H. erectus specimens, they were pooled for the statistical tests in Table 2.

C 1 distal shoulder height (no. 1 in Table 2). Canines of H. sapiens often exhibit an elevated distal shoulder that gives an incisor-like appearance to its crown. We metrically examined this character using the following index: distal shoulder height / labiolingual (LL) crown diameter. The height is the minimum distance between the distal cervical line and the distoincisal corner of the crown. The resulting index values were: 38% (OH 7: H. habilis); 50% (Sangiran 22) and 57% (Sangiran 7–58) (early Javanese H. erectus); 48% (Sinanthropus 70: Chinese H. erectus); and the modern human specimens range between 54% and 110% with the mean value being 78.4% (N = 109). When we categorized each specimen as having a ‘low’ or ‘high’ distal shoulder with the cut-off point of the index value set at 57.5, all three Early Pleistocene Homo C 1 s (as well as the Middle Pleistocene Chinese H. erectus C 1 ) were categorized as ‘low’ whereas only three out of the 109 H. sapiens specimens exhibit this primitive condition (Table 2). The frequency for each of the archaic Homo samples was significantly different from that in H. sapiens, suggesting that a H. sapiens-like spatulate C 1 appeared or became dominant after H. erectus/ergaster. Other East African Early Pleistocene C 1 s (OH 13, OH 16, KNM-ER 992) apparently had similarly low distal shoulder morphology although metric data are not available for these worn or damaged specimens. The single H. floresiensis C 1 available for this metric comparison (LB6/1: Fig 1G), whose distal shoulder remains unworn, also shares this primitive morphology (index = 52). The distal C 1 shoulder height for another H. floresiensis individual (LB1) cannot be measured due to wear, but the rounded mesial aspect and the convergent distal aspect of the occlusal outline of its exposed dentine (Fig 1A and 1B) suggests that the same also applies to this individual. In the H. sapiens sample, there is a weak correlation between the crown size (square root of the computed crown area) and the distal shoulder height index (r = −0.408, or −0.400 if logarithmic transformations are made) so that a smaller C 1 tends to have a relatively higher distal shoulder. The H. floresiensis C 1 s retain the low distal shoulder morphology despite being smaller than the modern human average in its LL diameter (Fig 7A).

P3 and P4 transverse crest (nos. 2 and 3 in Table 2). A transverse crest on a maxillary premolar was recorded when the longitudinal groove does not continue as a deep, continuous groove at the bottom of the occlusal surface but is more or less interrupted by a crest formed by occlusal ridges of the buccal and lingual cusps (corresponding to the category 2 of ref. [59]), regardless of its position (i.e., mesial, middle, or distal on the longitudinal groove). Such a crest formation is rare in our global H. sapiens sample (2% for both P3 [N = 283] and P4 [N = 279]) as well as in an European-dominated H. sapiens sample studied by Martinón-Torres et al. [65] (N = 124−132), but relatively common in the Early Pleistocene Homo (35% [P3] and 43% [P4] for our pooled Early Pleistocene sample) with significant differences between many pairs between the H. sapiens and the Early Pleistocene samples (Table 2). The P3 and P4 of H. floresiensis (LB1: Fig 1C) both exhibit this primitive trait (both on the enamel and EDJ surfaces) that is shared with H. habilis and early Javanese H. erectus.

P3 buccal grooves (no. 4 in Table 2). Premolars of Australopithecus and earlier Homo variably express vertical grooves on the mesial and/or distal aspects of their buccal faces [40,55,58]. Here, we compared the frequencies of appearance of one or both of these buccal grooves as opposed to their total absence. Only distinct grooves were counted; specimens with faint grooves/furrows or occlusally situated short grooves were recorded as absent to include slightly worn teeth. Buccal groove(s) are present in all the H. habilis P3s examined (N = 12), but 30‒40% or more of the P3s lack these grooves in later Homo groups, with the observed frequencies significantly different in many pair-wise comparisons between the H. habilis and the later Homo samples. In our sample, groove-free P3s appear only after H. habilis. Thus, the total absence of P3 buccal grooves in H. floresiensis (LB1: Fig 1C) is a derived condition at least relative to H. habilis.

P4 lingual crown development (no. 6 in Table 2). The previous geometric morphometric analyses [49] demonstrated that the most marked change in the crown shapes of the Homo maxillary premolars is reduction of the lingual crown. Australopithecus afarensis, Au. africanus, and H. habilis often exhibits a primitive configuration of MD pronounced lingual cusp that is equal to or slightly exceeds the buccal cusp, whereas a lingually tapering crown shape is a typical observation in more recent Homo [55,74–76]. Although we did not include the LB1 maxillary premolars in our EFA, the above known trend can be examined in part for P4 by a simple comparison between the buccal and lingual crown MD diameters. In the present study, a P4 was recorded as ‘lingual crown MD extensive’ in Table 2 when its maximum MD dimensions were buccal ≤ lingual. A few H. sapiens specimens develop a large distal accessory tubercle that obscures the above crown configuration. These specimens were recorded as missing data. The result (Table 2) shows that the H. habilis, the Dmanisi Homo, and the early Javanese H. erectus samples include a significantly higher frequency of specimens with a well-developed lingual crown than in H. sapiens, consistent with the previous findings [49]. Interestingly, a considerable number (4/6) of the East Asian Middle-Late Pleistocene archaic Homo P4s exhibit the advanced, reduced lingual crown morphology (Zhoukoudian, Chaoxian, and Xujiayao). The H. floresiensis P4 (LB1: Fig 1C) shows a primitive ‘buccal = lingual’ configuration but is different from a more primitive ‘buccal < lingual’ configuration often observed in H. habilis (5/8) and Dmanisi Homo (2/2).

P 3 lingual cusp position (no. 7 in Table 2). This trait was assessed with reference to the protoconid apex and the axis of the mesial and distal protoconid crests [40]. The lingual cusp (metaconid) position was recorded as ‘mesial’ when it is located slightly mesial or opposite to the buccal cusp (protoconid). All nine P 3 s of H. habilis display this primitive pattern reported for Au. afarensis [40,61,77] and is associated with a MD spacious talonid, but the lingual cusp is located distally relative to the buccal cusp in a few post-habilis Homo specimens (Table 2). This latter pattern is found in Dmanisi Homo (D211 [right only], 2735 [right only]) and African H. ergaster (KNM-ER 992, KNM-WT 15000 [left only]; OH 22), as well as Zhoukoudian H. erectus (Zdansky P 3 ), suggesting that it emerged after ~1.75 Ma in Homo [24]. A distally located P 3 lingual cusp is also occasionally found in H. sapiens (28/197 = 14%), although its differences from the H. habilis sample is not significant probably because of the relative rareness of this morphology. The three H. floresiensis P 3 s (LB1, LB2/2, LB6/1) exhibit this derived, distally oriented lingual cusp placement [24] (Fig 1A, 1B, 1E, 1G and 1H).

P 3 mesiolingual beveling (no. 9 in Table 2). A hominin P 3 is usually associated with a moderately thick and high mesial marginal ridge (that is often continuous but is occasionally interrupted by a vertical groove) as well as a distinct anterior fovea (either in the form of a pit or a slit). The H. floresiensis P 3 s are unique showing poor development of both of these structures. Instead, their entire mesiolingual occlusal surface is flattened and finely wrinkled, and is beveled mesially and lingually [24] (Fig 1A, 1B, 1E, 1G and 1H). We failed to find a single specimen with comparable morphology in our P 3 samples of H. habilis (N = 9), H. ergaster (N = 6), Dmanisi Homo (N = 2), early Javanese H. erectus (N = 4), Chinese archaic Homo (N = 9), as well as H. sapiens (N = 214).

P 4 transverse crest (no. 11 in Table 2). Transverse crest on mandibular premolars were recorded based on the same criterion used for the maxillary premolars (nos. 3 and 4). The available small sample suggests that a P 4 transverse crest that intervenes in the longitudinal groove appeared in post-habilis grade Homo (Table 2). Homo ergaster, early Javanese H. erectus, and the Middle Pleistocene East Asian archaic Homo display a significantly higher frequency of occurrence of this crest than in H. habilis. However, the observed frequencies significantly decrease from these post-habilis archaic Homo groups to the H. sapiens. Thus frequent occurrence of P 4 transverse crest is probably a primitive feature shared among post-habilis archaic Homo. This crest is commonly observed also in the European Middle-Late Pleistocene archaic Homo (~86%) [65,66,68]. At least one H. floresiensis individual (LB6/1) and probably a second (LB15/1) show this condition that is derived compared to H. habilis (Fig 1F–1H).

P 3 buccal basal enamel swelling (cingulum) (no. 14 in Table 2). Buccal enamel of canine, premolar, and molar teeth of Australopithecus occasionally show cingulum-like basal swelling to form a distinct band along the cervical line [55,74,76,78,79]. We counted this primitive morphology in our Homo samples focusing on P 3 . Such a structure was found in OH 6, Sangiran 22, and Sinanthropus 70 (a specimen belonging to the BI mandibles). P 3 of D2735 also has a similar structure (Martinón-Torres, personal communication) although it was previously described as a feature associated with enamel hypoplasia [43]. None of the H. sapiens P 3 s we examined exhibit basal enamel swelling on their buccal faces (N = 207), but one of the three H. floresiensis P 3 s (LB1) clearly possesses this primitive character (Fig 1B).

P 3 root form (no. 15 in Table 2). Wood et al. [60] schematized variation and evolution of mandibular premolar root number and spatial arrangement in Paranthropus and Homo. Although Australopithecus lacks a clear evolutionary trend in premolar root configuration [80], Homo shows a general trend of root number reduction from two-rooted patterns (‘MB + D’ pattern for P3 and ‘M + D’ pattern for P4, respectively) to a single-root form through varying forms of fusion between the two root components [60]. In the present study, we compiled frequency data of bifurcated versus fused/single mandibular premolar roots from the literature [33,40,41,43,45,62,73,81,82]. When distinct mesial and distal root components are fused to each other along their buccal margins [80], the specimens were counted as two-rooted. The results in Table 2 show that this trait is polymorphic in most samples compared here [60], but the Early Pleistocene Homo samples show bifurcated roots significantly more than in the large global modern human sample studied by Shield [73]. Two of the three H. floresiensis individuals exhibit bifurcated P 3 roots that are arranged in ‘MB + D’ pattern (LB1, 6/1) and the other individual has a fused (Tomes’) root (LB2/2) [24]. Given the rareness of the two-rooted P 3 s in H. sapiens (4%, N = 599) [73], the comparatively high frequency of this morphology in H. floresiensis (2/3) probably reflects its primitiveness [15]. More detailed morphometric analyses are needed to further investigate this issue (e.g., refs. [83–85]).

M 1 cusp number (no. 17 in Table 2). The absence of a hypoconulid on the M 1 (four-cusped M 1 ) is a trait unknown in H. habilis, H. ergaster, Dmanisi Homo, early Javanese H. erectus [43], as well as the Middle Pleistocene archaic hominin dental collection from China [45,81,86], although there is a report that two mid-Middle Pleistocene M 1 s from Atapuerca-Sima de los Huesos, Spain, exhibit this morphology [65]. A four-cusped M 1 is also rare in modern humans [68], with an observed frequency of 3% in our global sample (N = 269) and 1% in a much larger sample (N = 6790: Appendix A in ref. [87]). Although this frequency is as high as 10% in modern Europeans and Northern Africans, the reported frequencies are much lower in modern human populations from the other regions of the world. Despite its rarity, the M 1 s from the two H. floresiensis individuals (LB1, 6/1) lack the hypoconulid and are four-cusped [24] (Fig 1D and 1I). In our modern human sample, 4-cusped M 1 s are significantly smaller than 5-cusped M 1 s (P = 0.01, two-tailed t-test). This relationship raises a possibility that the loss of the M 1 hypoconulid occurred independently in H. floresiensis associated with its unusual crown shortening (Figs 2 and 6A).

M 2 cusp number (no. 18 in Table 2). All of the fossil M 2 s examined in this study (the archaic Homo samples from the Early Pleistocene of Africa, Caucasus, and Java, as well as the Middle Pleistocene of East Asia) have five major cusps, although possible occurrence of four-cusped M 2 s in early Javanese H. erectus has recently been reported [35,88]. Four-cusped M 2 s are also reported for several individuals of the European Middle Pleistocene archaic Homo (Atapuerca-Sima de los Huesos) [65]. This condition is quite common in H. sapiens [63,68,87] (59% in Table 2). The M 2 s from two H. floresiensis individuals are four-cusped [24] (LB1, 6/1: Fig 1D and 1I). As in the case for the M 1 , the loss of a hypoconulid occurred probably in association with the crown size reduction [68]. Four-cusped M 2 s are significantly smaller than 5-cusped M 2 s in our H. sapiens sample (P = 0.000, two-tailed t-test). Scott and Turner [87] observed a similar association in their sample of Pima Indians.

M 1 mid-trigonid crest (no. 19 in Table 2). A mid-trigonid crest (terminology follows ref. [69]) that bridges between the protoconid and metaconid and borders the distal aspect of the anterior fovea is common in the M 1 s and M 2 s of Neanderthals and European Middle Pleistocene Homo but relatively rare in H. sapiens [65–67,89]. Martinón-Torres et al. [43,64] suggested that this crest characterizes Eurasian archaic hominins as compared to African H. habilis and H. ergaster(see also ref. [90]), although other researchers caution that this crest takes variable forms [67] and more detailed studies are needed than a simple presence/absence dichotomy. Still, our data indicate this simple method can be used to distinguish Homo taxa [43,64]. In our samples, this crest occurs significantly more often on the M 1 s of early Javanese H. erectus than in H. sapiens (P = 0.003, Fisher’s exact test). A distinct mid-trigonid crest is absent in the available seven H. habilis M 1 s, but present in the two Dmanisi M 1 s [43]. Although the presence/absence of a mid-trigonid crest is obscured in the LB1 M 1 by wear, the strong expression of a crest on its EDJ surface (Fig 1C; Ref. [24]) strongly suggests that the crest was originally present on its enamel surface [67,69]. Therefore, this H. floresiensis individual probably shares the crested M 1 with the Eurasian Early Pleistocene Homo groups.

P 3 relative size (no. 21 in Table 2). As demonstrated in the main text and Fig 2, H. floresiensis is unique in having a remarkably large P 3 . In this section, we further examine this trait by comparing the relative MD lengths of P 3 /[P 3 +M 1 +M 2 ]) [24]. The data in Table 2 show that none of the archaic Homo (N = 14) and H. sapiens(N = 188) specimens reach the high index values exhibited by the two H. floresiensis individuals, confirming the remarkable relative P 3 size in this species. Allometry does not explain this unique morphology of H. floresiensis because its tooth size is within the variation of H. sapiens (Fig 2), and the smaller-toothed H. sapiens tend to show smaller relative P 3 MD lengths than in the larger-toothed archaic Homo specimens (Table 2).

Molar size proportion (no. 22 in Table 2). During the course of Homo evolution, the posterior molars experienced more marked size reduction than in the first molar, resulting in an alteration of the molar size sequence from plesiomorphic ‘M1 < M2 ≥ M3’ to derived ‘M1 > M2 > M3’ [56,91]. In the present comparison, we compare percent increases of the ‘tooth crown size’ (square root of the calculated crown area [MD × BL]) from M 1 to M 2 ([M 2 −M 1 ]/M 1 ), and from M 2 to M 3 ([M 3 −M 2 ]/M 2 ). The results for the M 2 -M 3 size proportions are reported in S3 Table and S1 Text because we found no clear inter-group differences for this trait. As for the M 1 -M 2 size proportion, the results in Table 2 show that H. habilis exhibits a primitive pattern of M 1 < M 2 (5−20% increase). Early Javanese H. erectus is also close to this primitive condition (4−9% increase), whereas the pattern observed in two H. floresiensis individuals, M 1 ≈ M 2 (0% [LB1] and 1% [LB6/1]), is found in Dmanisi Homo, H. ergaster, East Asian Middle Pleistocene archaic Homo, and H. sapiens. Thus, the M 1 -M 2 size pattern exhibited by H. floresiensis appeared only after H. habilis. Our global H. sapiens sample shows a wide range of variation from −15% to +5% with a weak correlation with the crown size (M 1 + M 2 ) (r = 0.202). The values of the two H. floresiensis individuals are atypical for H. sapiens when this correlation is taken into consideration (Fig 7D).