Here’s a simple fact with an uncertain explanation: historically, hurricanes with female names have, on average, killed more people than those with male ones.

Kiju Jung from the University of Illinois at Urbana–Champaign made this discovery after analysing archival data about the 94 hurricanes that hit the US between 1950 and 2012. As they write, “changing a severe hurricane’s name from Charley to Eloise could nearly triple its death toll”.

Why?

The names certainly don’t reflect a storm’s severity, and they alternate genders from one to the next.

Jung team thinks that the effect he found is due to unfortunate stereotypes that link men with strength and aggression, and women with warmth and passivity. Thanks to these biases, people might take greater precautions to protect themselves from Hurricane Victor, while reacting more apathetically to Hurricane Victoria. “These kinds of implicit biases routinely affect the way actual men and women are judged in society,” says Sharon Shavitt, who helped to design the study. “It appears that these gender biases can have deadly consequences.”

But Jeff Lazo from the National Centre for Atmospheric Research disagrees. He’s a social scientist and economist who has looked into the public communication of hurricane risk, and he thinks the pattern is most likely a statistical fluke, which arose because of the ways in which the team analysed their data.

Let’s look at each of these arguments in turn.

First, Jung’s team asked nine people to rate the name of US hurricane on a scale of 1 (very masculine) to 11 (very feminine). They found that the more feminine names were linked to more damage (normalised to today’s value) and deaths. (They excluded Katrina because that was such a huge outlier.)

To test their hypothesis about gender biases, the team ran six experiments. (For stats junkies, here’s the table showing all the numbers behind the experiments; note that each one involves a fresh group of volunteers.)

When the volunteers saw a list of hurricane names, and nothing more, they guessed that male storms would be more intense than female ones. After reading a more detailed scenario about an incoming hurricane, they predicted that the storm would be riskier and more intense if its name was Alexander rather than Alexandra.

After reading another similar scenario, they were more likely to say that they would evacuate their homes if Hurricane Christopher was hypothetically bearing down upon them, than if Hurricane Christina was doing so. Likewise, if they read a voluntary evacuation order, they were more likely to comply in the face of Hurricanes Danny, Victor or Alexander than Hurricanes Kate, Victoria, or Alexandra respectively

These differences aren’t due to explicit sexism. When the team asked people directly if male or female hurricanes would be more dangerous, the responses were evenly split. “This suggests that the effects in the main experiments are implicit in nature,” says Shavitt. In other words, gender stereotypes influence our thoughts and behaviour, whether or not we buy into them outright.

But Lazo thinks that neither the archival analysis nor the psychological experiments support the team’s conclusions. For a start, they analysed hurricane data from 1950, but hurricanes all had female names at first. They only started getting male names on alternate years in 1979. This matters because hurricanes have also, on average, been getting less deadly over time. “It could be that more people die in female-named hurricanes, simply because more people died in hurricanes on average before they started getting male names,” says Lazo.

Jung’s team tried to address this problem by separately analysing the data for hurricanes before and after 1979. They claim that the findings “directionally replicated those in the full dataset” but that’s a bit of a fudge. The fact is they couldn’t find a significant link between the femininity of a hurricane’s name and the damage it caused for either the pre-1979 set or the post-1979 one (and a “marginally significant interaction” of p=0.073 doesn’t really count). The team argues that splitting the data meant there weren’t enough hurricanes in each subset to provide enough statistical power. But that only means we can’t rule out a connection between gender and damage; we can’t soundly confirm one either.

Other aspects of the team’s analysis didn’t make sense to Lazo. For example, they included indirect deaths in their fatality counts, which includes people who, say, are killed by fallen electrical lines in the clean-up after a storm. “How would gender name influence that sort of fatality?” he asks. He also notes that the damage a hurricane inflicts depends on things like how buildings are constructed, and other actions that we take long before a hurricane is named, or even before it forms.

Then, there are the six experiments. As is common in psychology, the volunteers in the first three were all college students. “There is no reason to think that University of Illinois undergraduate students in hypothetical scenarios would have any relation to real-world decision making to populations in hurricane vulnerable areas,” says Lazo. The participants in the last three were recruited via Amazon Mechanical Turk—an online platform for finding volunteers. Again, it’s unclear how representative they were of people who live in coastal, hurricane-prone towns.

Finally, Lazo says that there’s a lot of evidence on how people respond to hurricane threats, and how their decisions are influenced by their social situation, vulnerability, culture, prior experience, sources of information, when the hurricane makes land, and so on. “Trying to suggest that a major factor in this is the gender name of the event, with a very small sample of real events, is a very big stretch,” says Lazo. And if the archival analysis isn’t as strong as it originally seemed, then what the team has basically done is to show “that individuals respond to gender”—hardly a big deal. [Update: To clarify, I mean that there’s already a huge amount of evidence that individuals respond to gender, not that the biases themselves are no big deal. Of course, they are.]

All of this matters because Jung’s conclusions, if they’re right, could have implications for how hurricanes are described. “It may make sense to move away from human names, but other labels could also create problems if they are associated with perceptions of mildness or gentleness,” says Shavitt. “The key is to provide information and labels that are relevant to the storm’s severity.”

Lazo says, “If there’s reasonable validity to their findings, it is worth further exploration with real hurricane-vulnerable subjects. That would be the proper conclusion to their study, and absolutely not any specific policy recommendations about changing naming conventions!”

Update: The authors have responded to Lazo’s criticisms in the comments below (see also this PDF). You can also find other critical viewpoints at Mashable, Slate, and indeed, in some of the comments in this piece.

Update 2: Bob O’Hara and GrrlScientist have written another rebuttal at the Guardian, pointing out flaws in the paper’s model. Check it out. I’d also pay attention to comments from Harold Brooks below.

Reference: Jung, Shavitt, Viswanathana & Hilbed. 2014. Female hurricanes are deadlier than male hurricanes. PNAS http://dx.doi.org/10.1073/pnas.1402786111. If the link isn’t working, this is why.