About a decade ago, evolutionary biologist Richard Merrill would spend several hours a day in “hot, steamy Panama,” sitting in a cage filled with Heliconius butterflies, waiting for them to have sex.

“Sounds glamorous, right?” he laughs.

Merrill was keeping track of whether male hybrid Heliconius butterflies would flirt—in the form of hovering or chasing—with either red-winged Heliconius melpomene rosina butterflies or white-winged Heliconius cydno chioneus butterflies. He documented this butterfly courtship to study the hybrids’ mate preference, which he and his team would later scrutinize at a genetic level.

In nature, hybrid Heliconius butterflies are rare. Heliconius melpomene and Heliconius cydno are both highly poisonous, having evolved to produce their own cyanide, and predators have learned exactly what both of these toxic insects look like. If the two species interbreed, however, their wing pattern becomes a disorienting mash-up of both color patterns, making the hybrid butterflies a stand-out target for predation. As a result, the hybrids’ lives often end before they can breed.

In a paper published yesterday in the journal PLOS Biology, Merrill and his colleagues have confirmed for the first time that the preferential mating behavior in these butterflies is indeed written in their DNA. Specifically, his team found just three parts of the genome that control at least 60 percent of mate choice behavior.

“It illustrates that a complex behavior like mate preference can be associated with just three regions of the genome,” says Erica Westerman, an evolutionary biologist at the University of Arkansas, who was not involved in the study. “This is something thought to be associated with lots of areas of the genome. It will allow us to take a targeted approach to see how these genes influence behavior of these butterflies.”

Exactly how these two Heliconius butterflies remain separate species is still something of a mystery. Speciation, or the process of creating a new species, is easily explained when there are geographic boundaries, like mountains, to physically divide a single species in two. What puzzles scientists is that H. melpomene and H. cydno have lived alongside each other in the same ecosystems, competing for the same resources, for more than one million years. Yet, the colorful, poisonous butterflies remain two separate species, refusing to mate and merge their genetic traits.

It’s a perfect demonstration of a biological concept called reproductive isolation, which has made Heliconius butterflies prime subjects for evolutionary studies for more than 100 years. Scientists hypothesize that reproductive isolation is maintained, in some cases, through powerful assortative mating, which means an organism will only reproduce with a mate that looks like them. Therefore, the barrier that divides them isn’t environmental, but rather genetic.

Merrill, who is now based at Ludwig Maximilian University of Munich, and colleagues used a method that shows which areas of the genome have the largest effect on mating behavior, but they did not pinpoint the exact genes. Even so, their analysis was clear enough to show that one of the three regions that influences mating preference is near a gene called optix, which is known to control the vibrant red wing patterns in H. melpomene. (Optix has such a strong influence on color that turning it on and off using the gene-editing tool CRISPR can make a butterfly’s wings totally colorless.) In fact, this genetic region is only 1.2 centiMorgans—which are units used to measure the distance between genes on a chromosome—away from the optix gene.

The fact that the genetic strings controlling mate preference are so close to optix, the gene that creates the wing patterns as well as some other visual cues, has exciting implications for researchers studying the evolution of behaviors like mating preference.

“[This study] provides a lot of insight into how preference and preference cue are linked physically,” says evolutionary biologist Susan Finkbeiner of the University of Chicago, who was not involved in the study. The research supports the idea “that forewing color pattern and preference for that particular forewing color are associated with one another.”

If the mating behavior and the preferred trait are indeed physically entangled on a single chromosome, then they would both be passed on to the next generation with ease, maintaining a genetic barrier of sorts between the two species. “We can have evolution of new species without evoking physical barriers, like seas or mountains,” Merrill says.

A second study showed that even though hybrid survival is rare, it's happened enough over the last million years that these two butterflies share ten times more genetic material than humans and Neanderthals. Even a few interbreeding events, it seems, can have a strong effect on genetics.

The research, led by evolutionary biologist Simon Martin of the University of Cambridge, used whole genome sequencing of nine populations of Heliconius to pinpoint areas of the butterflies’ DNA where hybridization and natural selection have influenced the organisms’ genetics over time. The fact that the species remain so visually distinct despite highly similar genomes reaffirms how powerful evolutionary forces are in shaping the tree of life.

“There’s not just one evolutionary path,” Martin says. “It’s a network or a web. But my study shows that it’s predictable. There’s a beautiful predictable pattern in this complex web of life.”

Ultimately, Martin’s results, also published in PLOS Biology, strengthen Merrill’s findings as well, showing that the two species remain separate because of the strong genetic barriers within their DNA that arose through natural selection—barriers like the connection between optix and reproductive preference. This bond between wing color and mate preference cannot be lost even in hybrid butterflies because the two genetic traits are so closely linked—possibly even driven by the same genes. Such genetic barriers make speciation predictable despite evidence of historic hybridization events.

“There’s predictability because of natural selection,” Martin says. “It’s not only in the creation of species but also in determining which genes are passed on and which aren’t [that] elevates the role of natural selection in evolution.”

As a next step, Merrill hopes to find the precise genes underlying this wing-color-preference behavior. He’s mulling over possibly using machine learning and video cameras to enable the team to collect more data next time around.

“We’re trying to develop methods to automate this process,” Merrill says. Once the team has specific genes to target, they can use CRISPR to do knockout studies and observe how butterflies behave without the genes that are thought to control their behavior.

Without the genes that control Heliconius butterflies’ choosy wing-color preference, the two separate species might be more inclined to mate with one another. To find out for sure, though, Merrill may have to head back to the butterfly cage at the Smithsonian Tropical Research Institute and sit and wait to see if he can catch any interspecies insect action. Although, he wouldn’t mind.

“There’s nowhere else in the world that you could have done this study,” he says.