Yesterday’s 5.8-magnitude earthquake centered in Virginia was notable for three main reasons. First, earthquakes of this magnitude are incredibly rare on the east coast; second, it proved yet again that Twitter moves faster than seismic waves*; and finally, one of Amazon’s AWS (EC2/S3) data centers is based in north Virginia, about 60 miles north of the quake’s epicenter.

Amazon’s data centers are positively packed with hard drives — thousands of them — and every single one of them is incredibly sensitive to vibrations. Generally, though, the only time a rack of servers moves is during an earthquake — and when this happens, the read/write heads are instantly thrown out of position, and accelerometers then kick in to prevent data loss. As a result, it suddenly becomes very hard to access data on the disk, and with Amazon AWS this means that page load/response times ballooned. It is for this very reason that west coast (and Japanese) data centers usually use base isolation to earthquake-proof their racks.

The graph on the right comes from SeatGeek, which hosts its site with Amazon AWS. Each of the vertical lines is two minutes — and so we can surmise that the earthquake was strong enough (and long enough) to affect the AWS hard drives for a couple of minutes. It’s also possible that the smaller spike on the left side of the graph was an early tremblor — but because the graph has been cropped (and its axes removed!) we don’t know if any aftershocks affected AWS.

Without an exact timeline it’s also impossible to say whether this spike was directly caused by the earthquake — but with a duration of just two minutes, it’s unlikely that it was because of increased traffic; a burst of traffic would last longer, as everyone and their mother starts hitting Twitter, Facebook, and news websites. On the other hand, it’s safe to assume that this same spike would be felt by every east coast data center — but lots of anecdotal evidence suggests that the web itself didn’t seem to slow down during the quake. Surely, if almost every hard drive in and around Washington DC was taking 10 times as long to access data, we would’ve noticed it?

Perhaps everyone was simply too busy tweeting to notice the slowdown, though. Last month, an ExtremeTech reader left a comment with a link to an amazing video (embedded below) that all but confirms that yesterday’s earthquake crippled Amazon AWS. In the video, a charismatic Australian (Brendan Gregg) shouts into a rack of hard drives, and the vibrations caused by his only-slightly-louder-than-ambient hollering is enough to slow down disk I/O significantly.

In the accompanying blog post, Gregg also notes that even a missing screw in a mounting bracket can introduce enough vibration to cause intermittent spikes in access time — so if you’re a serious gamer, remember to make sure your hard drives are mounted properly with anti-vibration grommets, and if you have a few thousand spare dollars, invest in one of those base isolation systems for your desktop rig, too. Alternatively, you could always replace your hard disk with a solid-state drive; moving parts are so passé, after all.

* XKCD “Seismic Waves” [Back to top]