Last weekend, I patiently sat in my tiny downstairs bathroom with my three daughters and a laptop during two sequential tornado alarms. We followed the storm on twitter until it was safe to emerge. My husband didn’t fit in the bathroom, so he kept his eyes and ears ont he storm outside, occasionally complaining about high tornado warning false alarm rates.

My husband and I both have roots in the Midwest and are used to frequent tornado warnings. In Illinois, where sirens sounded during every tornado warning. Before I had kids, I usually ignored the alarms unless it looked or sounded ominous outside. I take shelter these days, at least when I know about the alarms. We don’t have tornado sirens in Virginia, so we often miss them. This makes me uneasy, since there are a lot of tornadoes in Virginia, even though Virginia isn’t in “tornado row.” But TV warnings and social media are usually sufficient for alerting me during warnings.

Tradeoffs.Tornado warning are supposed to be conservative, meaning that the false alarm rate is high in order to ensure a low false negative rate. The false negatives–the alarm not signaling before a tornado strikes–can be deadly. I’m OK with a lot of false alarms so long as I can keep my kids safe. There are social costs to have too many false alarms in order to drive the false negatives to zero. Namely, we will all have in our storm shelter during a perpetual tornado warning. There are social costs with having a reasonable number of tornado warnings. The main issue here is that false alarms are akin to “crying wolf”–tornado warnings are eventually ignored, leading to people being in danger when there really is a tornado.

It turns out that there is some research on this topic.

Error rates. First, I would like to note that there is about a 7% false negative rate. It’s not as low as I would like, but it’s certainly quite low considering the rarity of tornadoes. No system has perfect sensitivity and specificity. Now let’s discuss false alarms. The National Weather Service defines a false alarm ratio (FAR):

FAR = unverified warnings / (verified warnings + unverified warnings)

A paper by Kevin Simmons and Daniel Sutter in the Weather, Climate, and Society (an American Meteorological Society journal) studied areas with lower and higher false alarm rates. They found:

A statistically signiﬁcant and large false-alarm effect is found: tornadoes that occur in an area with a higher false-alarm ratio kill and injure more people, everything else being constant. The effect is consistent across false-alarm ratios deﬁned over different geographies and time intervals. A one-standard-deviation increase in the false-alarm ratio increases expected fatalities by between 12% and 29% and increases expected injuries by between 14% and 32%. The reduction in the national tornado false-alarm ratio over the period reduced fatalities by 4%–11% and injuries by 4%–13%. The casualty effects of false alarms and warning lead times are approximately equal in magnitude, suggesting that the National Weather Service could not reduce casualties by trading off a higher probability of detection for a higher false-alarm ratio, or vice versa.

The April 27, 2011 tornado outbreak in Alabama cost many lives and resulted in some discourse on the false alarm rate and what people are willing to live with. A meteorologist from Alabama argues for fewer tornado alarms:

I firmly believe apathy and complacency due to a high false alarm ratio over the years led to inaction in many cases that could have cost lives. The FAR (false alarm ratio) for many NWS offices when it comes to tornado warnings is in the 80-90 percent category. I say this is simply not acceptable. Sure, the POD is excellent (probability of detection), but if most of the warnings are bad, then what good is a high POD?

blog post in the Washington Post further discusses the perils of high false alarm rate. The problem, however, has to do with how people react in response to false alarms, not with the high false alarm rates. I do wonder if we should be optimizing people instead of trying to optimize the false alarm rate. Public awareness campaigns could be helpful here. I don’t know how worthwhile this would be, since I have endured hundreds of tornado alarms and ignored many of them in my youth despite the many public service announcements in the Midwest. Perhaps TV stations could frequently air the movie Twister instead.

Discussing false alarms is silly if no one actually knows about the warnings. I have accidentally ignored tornado alarms when I am unplugged due to a lack of sirens in Virginia. My university doesn’t have tornado sirens. After the 2007 Virginia Tech shootings, a campus alert system was installed. A siren was installed to warn of campus dangers. The alarm sounds exactly like a tornado siren. It is even tested once a month, just like the tornado sirens in the Midwest (first Tuesday of the month at 10am in Champaign-Urbana, IL). I wondered why the siren couldn’t be dual purpose, since a back of the envelope calculation suggested that my risk of dying in a tornado on campus was much higher than my risk of during in a mass shooting event on campus. Someone else must have drawn the same conclusion, because the campus siren was used to signal a tornado warning on campus last week (yet another tornado alarm last week). The only problem then was that Virginians didn’t know what to do or where to go during a tornado warning (review what to do here ). In this day and age, tornado warnings can be easily sent via text message or via social networking, meaning that the perceived false alarm rate could actually be higher than what it used to be.

Do you take shelter during tornado warnings? How would you balance tornado warning true and false alarms?