ANN ARBOR—Did different species of early humans interbreed and produce offspring of mixed ancestry?

Recent genetic studies suggest that Neanderthals may have bred with anatomically modern humans tens of thousands of years ago in the Middle East, contributing to the modern human gene pool. But the findings are not universally accepted, and the fossil record has not helped to clarify the role of interbreeding, which is also known as hybridization.

Now a University of Michigan-led study of interbreeding between two species of modern-day howler monkeys in Mexico is shedding light on why it’s so difficult to confirm instances of hybridization among primates—including early humans—by relying on fossil remains.

The study, published online Dec. 7 in the American Journal of Physical Anthropology, is based on analyses of genetic and morphological data collected from live-captured monkeys over the past decade. Morphology is the branch of biology that deals with the form and structure of animals and plants.

The two primate species in the study, mantled howler monkeys and black howler monkeys, diverged about 3 million years ago and differ in many respects, including behavior, appearance and the number of chromosomes they possess. Each occupies a unique geographical distribution except for the state of Tabasco in southeastern Mexico, where they coexist and interbreed in what’s known as a hybrid zone.

The researchers found that individuals of mixed ancestry who share most of their genome with one of the two species are physically indistinguishable from the pure individuals of that species.

“The implications of these results are that physical features are not always reliable for identifying individuals of hybrid ancestry. Therefore, it is possible that hybridization has been underestimated in the human fossil record,” said Liliana Cortés-Ortiz, an evolutionary biologist and primatologist and an assistant research scientist at the U-M Department of Ecology and Evolutionary Biology and the Museum of Zoology.

First author of the paper is Mary Kelaita, a postdoctoral fellow at the University of Texas at San Antonio’s Department of Anthropology. The howler monkey study was part of Kelaita’s doctoral dissertation work at U-M’s Department of Anthropology.

For years, anthropologists have attempted to infer hybridization among human ancestral species based on the fossil record, which represents only a snapshot in prehistory, and have concluded that hybridization is extremely rare, according to Kelaita and Cortés-Ortiz. Given the utility of living primate models for understanding human evolution, the howler monkey study “suggests that the lack of strong evidence for hybridization in the fossil record does not negate the role it could have played in shaping early human lineage diversity,” Kelaita said.

The authors conclude that the process of hybridization (defined as the production of offspring through the interbreeding between individuals of genetically distinct populations), the factors governing the expression of morphology in hybrid individuals, and the extent of reproductive isolation between species should be given further consideration in future research projects.

In their study, Kelaita and Cortés-Ortiz analyzed different types of genetic markers, from both mitochondrial and nuclear DNA, to trace the ancestry of each howler monkey they studied. The use of molecular markers made it possible to approximate the relative genetic contributions of the parental species to each hybrid.

A total of 128 hybrid individuals were detected. Kelaita and Cortés-Ortiz found that most were likely the product of several generations of hybridization or of mating between hybrids and pure individuals.

Subsequently, they performed statistical analyses on body measurements and found a large amount of morphological variation in individuals of mixed ancestry. However, when individuals were classified according to the amount of their genome they shared with each parental species, it became clear that individuals of mixed ancestry that shared most of their genome with one of the species were physically indistinguishable from the pure individuals of that species. Even individuals that were more “intermediate” in their genetic composition were not completely intermediate in their appearance.

The study is the first to assess genetic ancestry of primate hybrids inhabiting a natural hybrid zone using molecular data to explain morphological variation.

Between 1998 and 2008, the researchers sampled 135 adult howler monkeys from Tabasco, Mexico, along with 76 others from Veracruz, Campeche, Chiapas and Quintana Roo states in Mexico and Peten in Guatemala. The field team collected blood, hair and morphometric measurements from the anesthetized animals before releasing them in the same locations. Sample collection from wild monkeys was carried out in accordance with U-M’s University Committee on Use and Care of Animals protocol #09319, and in collaboration with researchers at the Universidad Veracruzana in Mexico.

The animals were weighed, and 16 body-part measurements were made: trunk, tail, leg, foot, arm and hand length; chest and abdominal girth; head circumference and breadth; head, mandible and ear length; interorbital breadth; internasal distance; and testicular volume.

Howler monkeys are among the largest of New World monkeys, with male mantled howlers weighing up to 22 pounds. Fourteen species of howler monkeys are currently recognized. They are native to Central and South American forests, in addition to southeastern Mexico.

The study was supported by grants from the National Science Foundation, PROMEP-UVER Mexico, Universidad Veracruzana, and the U-M Office of the Vice President for Research, Museum of Zoology, Department of Anthropology and Rackham Graduate School.