A recently published paper claims that the gender of the name given to individual hurricanes is linked to the public's perception of the risk posed by that storm. In short, this study claims that hurricanes given female-sounding names are perceived to be less dangerous than those given male-sounding names (which we refer to here as "himmicanes"). This public underestimation of risk apparently results in hurricanes causing significantly more deaths than himmicanes.

Yesterday, we learned that hurricanes with female-sounding names are deadlier than those with male-sounding names. This cause quite a bit of astonishment and confusion (Bob's office mate's reaction to this news was typical: "Huh?"). In short, this study's results just sound barmy. But the authors have data, and they were kind enough to make that data publicly accessible to the unwashed masses (even though the paper itself is paywalled).

This gives us the opportunity to remind scientists that data sharing is a good thing, because it means the rest of us can check the authors' data analysis. But we also recognise that such intellectual generosity can lead to bruised egos ...

Anywho, we both are members of that crowd of skeptics whose bullshit detectors were twitching madly, but one of us (BOH) is a professional numbers torturer (a biostatistician) and for that reason, has pretensions to being numerate. So BOH downloaded and re-analysed the raw data. The method used was startlingly simple: (1) start with the authors' model, and then do what BOH was taught to do in his undergraduate stats course (2S1 at Leeds Uni, in 1990): check the model fit. [NOTE: if you want to explore further, you are welcome to read the full details, including R code on RPubs (here) or on GitHub (here).]

In short, the model that the authors fitted to the data sought to explain the number of deaths caused by a hurricane based upon (1) how "feminine" its name was perceived to be (this was measured subjectively by a panel, but really, just using the name's gender gives the same results), (2) the strength of the hurricane (measured by its minimum air pressure), and (3) the amount of damage caused. The authors then examined how the effects of strength or damage were moderated by name gender which then allowed them to answer the question: was the effect of strength, for example, on human deaths bigger if the hurricane's name was perceived to be more feminine?

According to their results, yes. (It's important to point out that there was no difference between average strengths or damages based on the gender of a storm's given name, but instead, stronger hurricanes caused more deaths if they had more feminine names.)

But this is a statistical model, which is a summary of the available data, and for that reason, we hope it is a reasonable representation of the data that it is based upon. How well does the model describe the actual data? At this point, we use a statistical technique that has been honed over the centuries (an early exponent of this method was Florence Nightingale) known as "drawing graphs". In particular, we plot the residuals -- basically, how the data deviate from the fitted model -- against the different covariates. After a bit of prodding, BOH uncovered a strong pattern in the effect of the amount of damage: the authors of the paper assumed this pattern followed a straight line, but actually, it doesn't (figure 1; larger view):

This isn't just an effect of hurricane Sandy: as a whole, the residuals are curved. So we must find a better model for the data. The obvious thing to do is to add a curve to the effect: in particular, a quadratic term (i.e.; we square the effect of damage and add that into the model. In fact, what BOH did was a bit more complicated: he ended up using the square root of the normalised damage). Using this approach made the residuals look much better (figure 2; larger view):

When we compare the data to the model itself, the "femininity effect" of hurricane names completely disappears: the only predictor of deaths is the normalised damage (as both linear and quadratic effects).

This BOH model assumes that the relationship between damage and mortality isn't a straight line, contrary to what the authors assumed in their paper. And why should it be? More to the point: is it more reasonable to assume that the relationship between damage and mortality isn't a straight line, or that hurricane name gender affects deaths? This is not a trivial question, and is better answered by experts in the field, rather than by statisticians (like BOH) or evolutionary biologists (like GS).

Of course, BOH is not claiming that his model is correct, either. But is this BOH model better than the model in the paper? Well, our BOH model does have two advantages: (1) it fits the data better and (2) it is more plausible.

The take-home message of this exercise is that data analysis is not a trivial matter. We can fit models to data, but they may not fit the data well. We all like to use complex sexy methods, but it's important to check that they actually describe reality. A little bit of checking early on can save a lot of embarrassment later.

Source:

Jung K., Shavitt S., Viswanathan M. & Hilbe J.M. (2014). Female hurricanes are deadlier than male hurricanes, Proceedings of the National Academy of Sciences, doi:10.1073/pnas.1402786111 [$]

NOTE [5 June 2014, 1333]: the data images in this piece were replaced with images that use stronger colours to make it easier for you to identify gender-name for each data point. The original images can still be viewed here: figure 1 and figure 2).

.. .. .. .. .. .. .. .. .. .. ..

Bob O'Hara is a biostatisician who conducts research at the Biodiversity and Climate Research Centre in Frankfurt am Main, Germany. He writes the blog, Deep Thoughts and Silliness at Occam's Typewriter. You can also follow him on twitter: @BobOHara.

GrrlScientist is an evolutionary biologist who also writes Maniraptora. She's very active on twitter @GrrlScientist and sometimes lurks on social media: facebook, G+, LinkedIn, and Pinterest.