In the mid-19th century, researchers claimed they could tell the sex of an individual just by looking at their disembodied brain. But a new study finds that human brains do not fit neatly into “male” and “female” categories. Indeed, all of our brains seem to share a patchwork of forms; some that are more common in males, others that are more common in females, and some that are common to both. The findings could change how scientists study the brain and even how society defines gender.

“Nobody has had a way of quantifying this before,” says Lise Eliot, a neuroscientist at Chicago Medical School in Illinois who was not involved in the study. “Everything they’ve done here is new.”

As soon as scientists could image the brain, they began hunting for sex differences. Some modest disparities have been reported: On average, for example, men tend to have a larger amygdala, a region associated with emotion. Such differences are small and highly influenced by the environment, yet they have still been used to paint a binary picture of the human brain, “even when the data reveal much more overlap than difference between males and females,” Eliot says.

So in the new study, researchers led by Daphna Joel, a behavioral neuroscientist at Tel Aviv University in Israel, tried to be as comprehensive as possible. Using existing sets of MRI brain images, they measured the volume of gray matter (the dark, knobby tissue that contains the core of nerve cells) and white matter (the bundles of nerve fibers that transmit signals around the nervous system) in the brains of more than 1400 individuals. They also studied data from diffusion tensor imaging, which shows how tracts of white matter extend throughout the brain, connecting different regions.

The team found a few structural differences between men and women. The left hippocampus, for example, an area of the brain associated with memory, was usually larger in men than in women. In each region, however, there was significant overlap between males and females; some women had a larger or more male-typical left hippocampus, for example, while the hippocampus of some men was smaller than that of the average female.

To accommodate this overlap, the researchers created a continuum of “femaleness” to “maleness,” for the entire brain. The male end zone contained features more typical of males, and the female end zone contained the version of the same structures more often seen in females. Then, the team scored every individual region-by-region to find out where they fell on that male-to-female continuum.

The majority of the brains were a mosaic of male and female structures, the team reports online today in the Proceedings of the National Academy of Sciences. Depending on whether the researchers looked at gray matter, white matter, or the diffusion tensor imaging data, between 23% and 53% of brains contained a mix of regions that fell on the male-end and female-end of the spectrum. Very few of the brains—between 0% and 8%—contained all male or all female structures. “There is no one type of male brain or female brain,” Joel says.

So how to explain the idea that males and females seem to behave differently? That too may be a myth, Joel says. Her team analyzed two large datasets that evaluated highly gender stereotypical behaviors, such as playing video games, scrapbooking, or taking a bath. Individuals were just as variable for these measures: Only 0.1% of subjects displayed only stereotypically-male or only stereotypically-female behaviors.

“There is no sense in talking about male nature and female nature,” Joel says. “There is no one person that has all the male characteristics and another person that has all the female characteristics. Or if they exist they are really, really rare to find.”

The findings have broad implications, Joel says. For one, she contends, researchers studying the brain may not need to compare males and females when analyzing their data. For another, she says, the extreme variability of human brains undermines the justifications for single-sex education based on innate differences between males and females, and perhaps even our definitions of gender as a social category.

The work “contributes in an important way to the conversation,” says Margaret McCarthy, a neuropharmacologist at the University of Maryland School of Medicine in Baltimore, who studies gender biases in neurological and mental health disorders. But she disagrees that it might not be useful to consider sex as a variable when studying the brain. She looks at rodent models to evaluate, for example, why males are five times as likely to develop autism, or why females are twice as likely to suffer from depression. “By studying male versus female brains, we have a great tool for exploring the biological basis of those differences,” she says. “[Joel’s] call for us to abandon the monikers of male and female or men and women I think is too far.”