We point to three caveats with respect to our computational model. In our model, mutation changes the haplogroup identity of individuals. This is not a good representation of real Y-chromosome mutations, which only change the haplotype of Y-chromosomes. Haplogroups, on the other hand, are clades of related Y-chromosome haplotypes defined post-hoc by researchers through the naming of nodes on a tree; the concept of a ‘haplogroup mutation’ has no counterpart in real life. However, we face computational limitations when attempting to model the dynamics of haplotypes, which are extremely diverse and are shared by at most several individuals out of a population of millions. We utilize the intuition that only a subset of the mutations occurring within the ancient population we are simulating lead to clades of Y-chromosomes that become named as haplogroups by researchers who observe the phylogenetic tree at a later date. To implement this, we use a ‘haplogroup mutation rate’, or the rate at which haplogroup-defining mutations occur, which is much lower in our model than the Y-haplotype mutation rates found in the literature. We discuss approaches that may resolve this discrepancy in Supplementary Note 6.

A second caveat for our model is that patrilineality is treated exogenously. It is treated as a preexisting distinction between two sets of social ecologies whose origins are left unexplained. While the origin of unilineal descent groups is difficult to model, anthropologists have suggested various hypotheses, ranging from social adaptation to the presence of heritable wealth and accumulations of property49, or cultural evolution under conditions of intergroup competition48. Modelling the evolution of the causative social structures could form the basis for an extension of the current work.

The third caveat concerns the possibility that the cultural groups exhibit spatial structure, which our model does not address. Stepping stone dynamics for both competition and group fusion, which is a more realistic representation of population ecology, should result in haplogroup similarity being correlated with geographical distance between cultural groups. Supplementary Note 4 includes a ‘stepping stone’-type fusion of cultural groups as a modification of our model.

Our proposal is supported by findings in archaeogenetics and anthropological theory. First, our proposal involves an episode in human prehistory when patrilineal descent groups were the socially salient and major unit of intergroup competition, bracketed on either side by periods when this was not the case. That such a sequence of sociocultural scenarios is plausible is supported by the extensive literature on the unilineal kin group in anthropology, which emphasizes its prevalence in societies with ‘mid-range complexity,’ and its absence or weakness in both the simplest and most complex societies (‘curvilinear trend’53, also see citations on page 2 of Korotayev47). Cross-cultural data suggests that unilineal kin groups became widespread with the increase in population density and ecological intensification associated with increasing reliance on agriculture, animal husbandry, or stationary resource concentrations (see Johnson and Earle50 pp. 45–51, 157–171), only to decline in importance or even disappear entirely with the emergence of complex regional polities, which create ‘conditions unfavourable for large unilineal kinship groups’53.

Indeed, Kelly49 finds that segmentary societies, or societies where ‘social substitutability’ of members of descent groups was apparent during conflicts, were generally small-scale societies with frequent warfare, and that such societies were either agropastoralists or sedentary hunter gatherers with food storage or exploiting stationary resource concentrations. Nomadic hunter gatherers with low population densities were not among this group. Ember et al.48, using a data from the Ethnographic Atlas, show that unilineal descent groups were always present in societies with endemic warfare but without centralized authority—i.e. in any non-state non-chiefdom society where war was present. Johnson and Earle50 and Kelly49 suggest that corporate kin groups, as means of social organization and as agents in intergroup conflict, may arise with increasing density, intensity of ecological exploitation, importance of capital investment and social circumscription, all factors which tend to increase in an agropastoralist landscape, and which also increase the importance of enforceable and excludable property rights. Intensive agriculture and pastoralism, by increasing the need for enforcement of social rights to resources, may have led individuals to agglomerate and incorporate along the most constant and least overlapping set of interpersonal ties, those of unilineal kinship. Indeed, unilineal descent groups are highly efficient at mobilizing for collective action, including intergroup competition54,55. Thus, the patrilineal corporate kin group may have become more prevalent due to its utility in competition and in guaranteeing access to vital resources after the intensification of environmental exploitation during the Neolithic transition.

On the other hand, the development of political complexity in chiefdoms and states tends to reduce the prominence of corporate kin groups. They may not entirely disappear, but their relevance as units of mobilization in intergroup competition must be reduced if sustained increases in social scale were to be achieved44,56. Increased market involvement under state peace also tends to catalyse the eventual breakup of descent groups in cultures with long histories of state rule47. Trajectories of political development in Madagascar57 and Polynesia58 provide some insight into the process by which kinship structures were steadily superseded as major units of violent competition following the emergence of the state or regional polity.

There is evidence that other analogous situations involving gene-culture hitchhiking in culturally-defined social groups may have affected genetic diversity. Central Asian pastoralists, who are organized into patriclans, have high levels of intergroup competition and demonstrate ethnolinguistic and population-genetic turnover down into the historical period59. They also have a markedly lower diversity in Y-chromosomal lineages than nearby agriculturalists42,60. In fact, Central Asians are the only population whose male effective population size has not recovered from the post-Neolithic bottleneck; it remains disproportionately reduced, compared to female estimates using mtDNA4. Central Asians are also the only population to have star-shaped expansions of Y-chromosomes within the historical period, which may be due to competitive processes that led to the disproportionate political success of certain patrilineal clans60.

Analogous phenomena are also found among nonhuman species. In whales, species with matrilineal organization such as Orcas, Pilot and Sperm whales have reduced mitochondrial diversity. The reduction is such that cultural selection operating at the group level61,62 has been invoked to explain the phenomenon.

Another line of evidence supporting our proposal comes from the shallow coalescence observed in phylogenies constructed from ancient DNA samples found in Europe. Archaeogenetic sequencing of samples from diverse cultures in Europe has enabled investigation of inter-population relationships11,63. Less investigated is the pattern of internal genetic relations between members of the same cultural group.

We use shallowness of coalescence, a qualitative feature of phylogenies, to describe how often coalescences occur in the tree of Y-chromosome sequences from archaeogenetic samples from single cultures. In cultures with deep coalescence, Y-chromosomes coalesce at very long timescales; in cultures with shallow coalescences, many men carry Y-chromosomes that coalesce a short time prior to their deaths. Cultures with deeper coalescences contain males with low levels of paternal relatedness; the converse applies for cultures with shallow coalescences. Figure 6 displays Y-chromosomal phylogenies of multiple samples from across Europe from cultures with both hunter-gatherer and agropastoral modes of subsistence.

Fig. 6 Tree of Y-chromosome genotypes from samples found among cultures with hunter-gatherer subsistence, and agropastoralist subsistence. The blue background represents hunter-gatherer subsistence while the green background represents agropastoralist subsistence. Letters in red circles match individuals from sites with their archaeological context. Note that R1b-P321 is synonymous with R1b-S116. Adapted from Figs. 3, 4, 5 and 6 of Kivisild67, with addition of information from Olalde et al.64. The vertical axis represents time; the position of branch points represent the ages of branch-defining mutations, with nomenclature and age from yfull (https://www.yfull.com/tree/) Full size image

Figure 6 shows a striking pattern of differences in shallowness of coalescence in samples from hunter-gatherer, farmer and pastoralist cultures. While hunter-gatherer Y-chromosomes from the same culture, and often the same sites, commonly divide into haplotypes that coalesce in multiple millennia, Y-chromosomes of samples from farmer and pastoralist cultures are more homogeneous and have more recent coalescences. The Bell Beaker culture has a high proportion of sampled males (81%) from a large geographical area (Iberia to Hungary) who belong to an identical Y-chromosomal haplogroup (R1b-S116), implying common descent from a kin group that existed quite recently. Some groups of males share even more recent descent, on the order of ten generations or fewer64. Such recent common descent may even be retained in cultural memory via oral genealogies, such as among descent groups in Northern and Western Africa, whose members can trace descent relationships up to three to four centuries before the generation currently living40. Likewise, from Germany to Estonia, the Y-chromosomes of all Corded Ware individuals sampled, except one, belong to a single clade within haplogroup R1a (R1a-M417) and appear to coalesce shortly before sample deposition.

Thus, groups of males in European post-Neolithic agropastoralist cultures appear to descend patrilineally from a comparatively smaller number of progenitors when compared to hunter gatherers, and this pattern is especially pronounced among pastoralists. Our hypothesis would predict that post-Neolithic societies, despite their larger population size, have difficulty retaining ancestral diversity of Y-chromosomes due to mechanisms that accelerate their genetic drift, which is certainly in accord with the data. The tendency of pastoralist cultures to show the lowest Y-chromosomal diversity and the shallowest coalescence would also be explained, as they may have experienced the social conditions that characterized cultures of the Central Asian steppes42. Indeed, the Corded Ware pastoralists may have been organized into segmentary lineages65, an extremely common tribal system among pastoralist cultures, including those of historical Central Asia66.

Additionally, the homogeneity in Y-chromosomes across the geographic spread of the extensive Bell Beaker and Corded Ware cultures may testify to continuity in patrilineal kinship even with group fission and long-distance migration, which may be consistent with a phenomenon observed in modern ethnographies of patrilineal segmentary cultures, that of ‘predatory expansions’54.

The rate of coalescence of Y-chromosomal haplogroups may not be fully captured by currently available Y-chromosomal genotyping, as most archaeogenetic samples are sequenced through hybridization capture methods, not whole-genome sequencing67. Such methods enrich for human-specific DNA in SNP (single-nucleotide polymorphism) arrays targeted towards phylogenetically relevant SNP sites in modern populations, which allows for the capture of phylogenetic information from samples with low endogenous DNA content. These methods tend to lose information from chromosomal regions far from SNP sites in the array, which may cause downstream mutations private to the ancient samples that unite them at phylogenetically more recent nodes to be missed. Therefore, Fig. 6 is only a preliminary demonstration of patterns in genetic data, and may change with data from ancient Y-chromosomes sequenced at higher resolutions.

Our hypothesis represents an attempt to synthesize genetic, anthropological and archaeogenetic data to create a synoptic view of social dynamics, or the social process, as a braided stream in time and space. The social mechanisms we describe in our hypothesis may have caused co-transmission of Y-chromosomes in social groups with developments in social structure after the Neolithic, causing an episode of massively increased gene-culture hitchhiking61 that affected only Y-chromosomes, which then ended with the emergence of complex societies. Our mathematical models support our hypothesis and illustrate the utility of combining data from genetics and archaeogenetics with theoretical models that approximate realistic social and anthropological scenarios. This represents a step forward in synthesizing the growing body of information emerging from ancient DNA studies. Indeed, future modelling approaches (see Supplementary Notes 5 and 6), together with the excavation and sequencing of greater numbers of archaeogenetic samples, will continue to put our hypothesis to the test.

Exploration of uniparental variation in modern human populations from cultures at different social scales or with different political histories, or from some geographic areas that have societies that never left the ‘bottleneck period’ (such as the Central Asian pastoralists from Karmin et al.), or from societies that never entered it, may represent further tests of our hypothesis. Some findings give us preliminary indications about the impact of historical social structure on genetic variation. For example, Irish men sharing the same surname, even very common ones, are approximately 30 times more likely to share a Y-chromosomal haplotype than a random pair of Irish men68,69. This is not the case for the English, who demonstrate much weaker signals of coancestry between Y-chromosomes and surnames, especially when the surname is very common70,71. This difference parallels contrasts in the relative importance of patrilineal clans in the social origins of surnames in the history of the two populations.

There are also significant variations in bottleneck intensity, with the bottleneck being less extreme in East Asian and Southeast Asian populations than in West Asian, European, or South Asian populations (see Fig. S4a, b of Karmin et al.4). The wider distribution and greater importance of pastoral cultures in the ancient Middle East, Europe and India may have played a role in creating this difference. This may imply that prehistoric interactions between farmers and herders in the Middle East and South Asia involved interesting social patterns, as in Europe. Farmer-herder interactions have been recognized as playing a key role in cultural change, such as in the creation of ideals of personal, mobile and alienable property72, or the rapid development of political complexity38.

At the same time, shallowness of coalescence may be useful in anthropology and archaeology, as it may allow for inference about social and political realities from archaeogenetic sequences. It may be used as a measure of intensity of intergroup competition and rates of social group extinction, if we have sufficient evidence that the cultures under investigation were patrilineal and patrilocal. Conversely, if the cultures were matrilineal and matrilocal, intense warfare may have produced negligible effects on Y-chromosomal diversity. The impact of warfare on mitochondrial diversity in matrilineal societies represents an interesting research question. Archaeogenetic sequences from around the world, including from societies that were inferred to be matrilineal, could be profitably investigated for insights into past social conditions.

Lastly, multiple characterizations of the ‘chiefdom’—conceived as a grade of political and social development—have emerged in the literature73, and our hypothesis has interesting implications for the interaction between kin groups and emerging social complexity. Many theorists assert that the transition from kinship-based to non-kinship-based mechanisms of sociopolitical organization was a critical episode in human social evolution44,56,74. The emergence of ideologies that help to legitimise and propagate a social class which cuts across kin groups would, in these theories, constitute a crucial juncture in the emergence of ‘ultrasocieties’ in which kinship is no longer the most important element in sustaining sociopolitical relations and coalition formation. Ideologies that support legitimate power, and the institutional positions or offices that possess such power, may now exist as a parallel set of cultural elements that, alongside kinship, also enable the creation of sociopolitical structures. As ‘conquest war’ replaces ‘competition war’50—a transition in accord with our hypothesis as well—the institutions and social formations created by such ideologies, as well as the ideologies themselves, may serve as the new target of cultural selection, instead of the populations involved in the competitive process.

If all ultrasocieties are the products of historical junctures where kinship organizations were supervened through cultural and ideological change, this should be supported by differences in patterns of uniparental genetic variation between cultural groups with different political histories. An investigation into the patterns of uniparental variation among, for example, the Betsileo highlanders of Madagascar, who may have undergone an entry and an exit from the ‘bottleneck period’ very recently57, could reveal phenomena relevant to such history. Cultural changes in political and social organization—phenomena that are unique to human beings—may extend their reach into patterns of genetic variation in ways yet to be discovered.