With hard disk drive (HDD) capacities edging upwards – 6TB HDDs are now available – Raid is becoming increasingly problematic as a method of data protection against hardware failure.

By submitting my Email address I confirm that I have read and accepted the Terms of Use and Declaration of Consent.

Please check the box if you want to proceed.

Please check the box if you want to proceed.

Enjoy this article as well as all of our content, including E-Guides, news, tips and more.

As a response, erasure coding as an alternative to backup has emerged as a method of protecting against drive failure.

Raid just does not cut it in the age of high-capacity HDDs. The larger a disk's capacity, the greater the chance of bit error.

And when a disk fails, the Raid rebuild process begins, during which there is no protection against a second (or third) mechanism failure. So not only has the risk of failure during normal operation grown with capacity, it is much higher during Raid rebuild, too.

Also, rebuild times were once measured in minutes or hours, but disk transfer rates have not kept pace with the rate of disk capacity expansion, so large Raid rebuilds can now take days or even longer.

Consequently, many argue that alternatives to Raid are now needed, and one alternative is erasure coding.

Erasure coding explained Erasure coding is a method of data protection in which data is broken into fragments that are expanded and encoded with a configurable number of redundant pieces of data and stored across different locations, such as disks, storage nodes or geographical locations. The goal of erasure coding is to enable data that becomes corrupted to be reconstructed by using information about the data that is stored elsewhere in the array – or even in another location. It works by creating a mathematical function to describe a set of numbers so that they can be checked for accuracy and recovered if one is lost. Otherwise known as polynomial interpolation or oversampling, this is the key concept behind erasure coding methods that are implemented most often using Reed-Solomon codes. Developed in 1960, Reed-Solomon is found most widely on CDs and DVDs, where error correction allows a player to calculate the correct information even though part of the disc's surface may be obscured. It is also used by space agencies to pick up signals from far-flung spacecraft, such as the Voyager probes.

Erasure coding use cases Erasure coding's high CPU utilisation and latency make it well suited to archiving applications because of the long-term nature of the storage where, over time, a number of storage elements can be expected to fail. It is also suited to those with large datasets and a correspondingly large number of storage elements. Staimer adds: "You could start thinking about erasure coding in the hundreds of terabytes, but once you get to a petabyte, you should definitely be thinking about it. And, once you get into the exascale range, you have to have something with erasure coding." Erasure coding is also found in the context of object storage, with very large-volume cloud operators the most likely users now. So, erasure coding is less suited to primary data and, like Raid, it cannot protect against threats to data integrity that are not a result of hardware failure. For applications where latency is not an issue, such as archiving, erasure coding works by ensuring that the life of the storage medium, which on its own can never offer a 100% guarantee for all time, is extended.