Lossless audio formats comparison

A comparison of six lossless audio formats.

Bobulous Central → Miscellaneous pages → Lossless audio formats 2006.

This is page two of a three-page article. This page covers lossless audio formats. Page one is the introduction page, which sets out the test method and lists the CDs used in the comparison. Page three covers lossy formats.

Lossless codecs and settings

This comparison examines six lossless audio formats, five of which use lossless compression. Compression modes were chosen based on recommendation by the codec's creator. Failing such a recommendation, I looked for the consensus on audio discussion forums. Failing that, I settled on the mode I felt produced the best compromise between speed and resultant file size.

Wave (.wav) Uncompressed Wave files are some people's idea of an ideal lossless format when archiving their CD collection digitally. Because the audio files are uncompressed, the file size is larger, but the rip-and-encode process should take less time. FLAC (.flac) dBpowerAMP codec release 5.3, using FLAC 1.1.2. Compression mode: 4 (medium). FLAC is an open source codec that produces lossless compressed files. It has been around a while, and it's the format I use when I'm archiving my CD collection. Even though lossless files are relatively large, there are a small number of portable music players that support playback of FLAC files. There is also support for FLAC in several digital hi-fi systems. FLAC files support tagging to store track details. Monkey's Audio (.ape) dBpowerAMP codec release 7, using Monkey's Audio 3.99. Compression mode: High. When I published my last audio format comparison, I was worried that development on Monkey's Audio had dried up. It turns out I was wrong; the Monkey's Audio codec has seen version updates twice this year, so it's still very much in the game. Monkey's Audio files use their own APE tagging system to store details about the audio file. OptimFROG (.ofr) Using OptimFROG 4.506. Compression mode: Fast. This codec has a reputation for strong compression, so I decided to give it a try. Shorten (.shn) dBpowerAMP codec release 3, using Shorten 3.5.1. Compression mode: lossless default. Shorten is seemingly very popular with people who trade in recordings of live music, and there are whole discussion groups dedicated to the subject of where to find SHN files of particular artists. WavPack (.wv .wvc) dBpowerAMP codec release 4, using WavPack 4.3.1. Compression mode: Fast. Performance tables seemed to put this codec's speed in the same neighbourhood as FLAC's, and I wanted to see for myself. WavPack is also an open source codec.

Raw data

Rip-and-encode times

Firstly, the time taken to rip each CD album and encode it using each lossless codec. The time recorded was that reported by dBpowerAMP after the rip-and-encode process of each album was complete.

Time in seconds needed to rip and encode each album with each codec Album Wave FLAC WavPack Shorten Monkey's Audio OptimFROG album mean average Life of Destructor 103 101 119 142 130 252 141.2 Exit Planet Dust 107 105 122 150 137 267 148.0 Demon Days 109 108 125 153 141 273 151.5 The W 123 123 151 176 162 313 174.7 Endtroducing. . . . . 126 127 153 179 176 328 181.5 The Best of Nick Cave... 136 146 192 203 204 381 210.3 Superunkown 140 146 187 208 209 381 211.8 Barrio Fino 140 150 198 209 212 392 216.8 Resident Evil: Apocalypse 135 151 196 211 211 398 217.0 The Essential Classics Collection 146 200 173 203 278 372 228.7 format mean average 126.5 135.7 161.6 183.4 186.0 335.7 188.2

Lossless file sizes

Next, the total file size of each album stored in each lossless format.

File size in bytes of each album encoded with each codec Album Monkey's Audio OptimFROG FLAC WavPack Shorten Wave album mean average Life of Destructor 300,483,192 301,805,204 316,803,249 319,344,246 335,870,042 487,835,816 343,690,292 Exit Planet Dust 314,072,235 314,773,558 331,367,604 335,924,925 371,818,203 523,075,876 365,172,067 The Essential Classics Collection 282,403,706 285,372,817 303,111,263 300,311,766 310,475,191 742,460,908 370,689,275 Demon Days 321,027,835 321,722,606 343,758,092 344,219,955 371,766,015 537,879,540 373,395,674 The W 352,109,763 353,662,913 372,014,851 373,411,869 434,884,909 626,462,828 418,757,856 Endtroducing. . . . . 357,560,360 359,459,746 382,116,673 383,168,158 420,184,040 667,505,360 428,332,390 Superunknown 500,597,960 503,973,822 522,006,755 525,194,210 557,709,496 779,745,152 564,871,233 The Best of Nick Cave... 492,129,385 498,955,693 519,281,439 526,254,421 552,536,492 800,224,016 564,896,908 Barrio Fino 544,066,012 546,697,374 569,206,029 570,641,928 593,960,352 808,338,768 605,485,077 Resident Evil: Apocalypse 554,499,706 559,473,643 576,383,453 582,654,498 609,737,886 806,715,624 614,910,802 format mean average 401,895,015 404,589,738 423,604,941 426,112,598 455,894,263 678,024,389 465,020,157

Lossless compression rates

Finally, a look at how well each album compressed on average. This is measured by dividing the average file size of each album encoded to a compressed lossless format (so we aren't including the uncompressed Wave format) and dividing it by the number of seconds in that album's running time. This gives us the average lossless compression rate for each album, in bytes per second.

Compressed file size in bytes per second of album running time Album Average compressed size Running time (hh:mm:ss) Average bytes per second The Essential Classics Collection 296,334,949 01:04:50 76,179 Endtroducing. . . . . 380,497,795 01:02:59 100,687 Demon Days 340,498,901 00:50:43 111,896 Exit Planet Dust 333,591,305 00:49:19 112,738 Life of Destructor 314,861,187 00:46:00 114,080 The Best of Nick Cave... 517,831,486 01:15:29 114,337 Superunknown 521,896,449 01:13:33 118,263 Barrio Fino 564,914,339 01:16:10 123,614 Resident Evil: Apocalypse 576,549,837 01:16:06 126,270 The W 377,216,861 00:49:04 128,131

Analysis of the data

Average rip-and-encode times

Overall, Wave is still the fastest lossless format to rip and encode a CD to. But I was shocked to see FLAC finish faster than Wave on three of the albums. FLAC took only seven percent longer than Wave, on average. A great improvement over the performance I saw from FLAC in my previous audio format comparison.

WavPack was next fastest, but couldn't keep up with FLAC. An extra twenty-five seconds taken for each album isn't too bad, unless you've got a big stack you want to rip and encode all in one go.

Shorten came in next, at an average of over twenty seconds longer than WavPack.

Monkey's Audio came in fifth, taking almost fifty percent longer to rip and encode the albums than it took to rip and encode to uncompressed Wave.

OptimFROG took an awful long time to encode each album. It was the slowest on every album tested, and slowest by quite some time. On average, it took OptimFROG two-and-a-half minutes longer to rip and encode a CD than Monkey's Audio, and over two-and-a-half times the amount of time taken to rip and encode to Wave. If OptimFROG had offered a faster compression mode, I'd have taken it. But Fast mode is the fastest on offer. Five-and-a-half minutes feels like a long time to rip and encode a single album.

Average album sizes

Monkey's Audio produced the smallest lossless file size for every one of the ten albums tested. And its performance seems to have improved a little since my previous format comparison. On average, an album encoded into Monkey's Audio takes up only fifty-nine percent of the space that the same album encoded into uncompressed Wave would.

OptimFROG comes in second, producing average album sizes only a little larger than Monkey's Audio, at sixty percent of the size used by Wave.

FLAC comes in third, and also seems to have improved its ability to compress since my last format comparison, this time producing an average album file size only sixty-two percent that of uncompressed Wave.

WavPack is very close behind FLAC, producing an average file size of sixty-three percent that of Wave. One interesting observation is that WavPack files took up slightly more space than FLAC for every album except The Essential Classics Collection. The smooth nature of classical music perhaps agrees more with WavPack than it does FLAC. The difference was still slight, however.

Shorten offered the least effective compression in this comparison. On average, the albums compressed with Shorten took up sixty-seven percent of the space that Wave used. This is roughly the performance that FLAC turned in during my last comparison.

Wave, being uncompressed, is not surprisingly the least space-efficient format. The albums tested in this comparison ended up using an average 678 MB per album when stored in uncompressed Wave format, which would take up almost seventy percent of a 1GB flash disk in a portable music player. (If anyone were mad enough to use uncompressed Wave on a portable player.)

Each format held the same order of effectiveness for every album except The Essential Classics Collection, which caused WavPack to be slightly more effective than FLAC for just that one album.

Average compression rates

The Essential Classics Collection (Volume 1) is by far the most successfully compressed of the ten albums. Its average lossless compression rate is 76 KB per second of the album. I assume this is because classical music is much slower and smoother than modern studio music, and because classical music features much more silence and near-silence. Silence ought to be extremely easy to compress for any codec.

Endtroducing is not as compressible as the classics, but it still needs fewer bytes per second than the other eight albums, and by some margin. This album features fairly orderly and laid-back tracks, as well as sparse verbal tracks that ought to be easy to compress.

Most of the remaining albums loiter in the 110–120 kilobytes per second range. But three albums demand more. Barrio Fino, Resident Evil: Apocalypse, and The W all need over 123KB per second. These albums all feature tracks that are full of noise that doesn't let up. Barrio Fino is thick with dance sounds, Resident Evil: Apocalypse vibrates with wailing guitars, and The W is an album throbbing with varying basslines and samples. Such densely-packed audio is apparently difficult to compress losslessly.

Conclusion

In my opinion, it's a choice between FLAC and Monkey's Audio. And the choice depends on whether you value time or disk space more.

FLAC is now so fast that the time saving you'd make by ripping and encoding to uncompressed Wave is barely worth thinking about. FLAC took the same amount of time or less than Wave on four of the ten albums. Combine that with the fact that encoding to FLAC will save you almost forty percent of the space that Wave uses, and you'd be mad to keep using Wave to rip and encode CDs on the basis of saving time.

If disk space is more important to you, then Monkey's Audio may be the codec for you. Ripping and encoding to Monkey's Audio will take you a third longer than doing so with FLAC, but Monkey's Audio produced the smallest file sizes for each of the ten albums tested here. The space saving is only five percent of the space taken by FLAC files, an average of 21 MB per album, but if you're on a tight disk budget, or if you want to serve the files from a web server, twenty megabytes per album might mean a lot to you.

OptimFROG and WavPack show promise. OptimFROG is clearly capable of compression of the same strength as Monkey's Audio, but it needs to become a lot faster before it will look attractive. WavPack offers the same sort of compression strength as FLAC, but isn't as fast. The speed of a software process usually improves as the software gets developed, as optimisations allow the same work to be done more efficiently. So these lossless codecs may well offer faster ripping and encoding times in the future.

Shorten performed, in my opinion, the least impressively of all the compressed lossless codecs, being the slowest and least effective at reducing file size. To all the people who trade in live music recordings stored in the Shorten format, I recommend they take a look at FLAC or Monkey's Audio, both of which will save space and bandwidth compared to SHN files. [Shorten's creators, SoftSound, now seem to have gone offline.]

Of course, if you need even bigger space savings, and couldn't care less how long it takes to encode your audio, then you should give the slower compression modes of some of the codecs a try. In this comparison I've gone with the compression mode I think will give the best mix of speed and size for each codec, but some people will care only about size.

Whichever codec you choose to encode your music with, make sure you test a few codecs for yourself first. That way you can decide which one produces the best compromise between speed and size.

For me, though, it's going to continue to be FLAC that I use to rip and encode albums to.

Evidence that conversions really are lossless

To make sure that the audio stored in each lossless format really is the same as the original audio source, I used an uncompressed Wave file of Canon in D major from The Essential Classics Collection. This Wave file was examined using the handy HashTab Shell Extension, and the MD5, SHA1 and CRC-32 hash values for the file were noted down, as was the size of the Wave file in bytes. This Wave file was converted using dBpowerAMP into a FLAC file (using the same compression settings as used in the big comparison above), and the original Wave file was deleted. The FLAC file was converted into a Monkey's Audio file, which was converted into an OptimFROG file, which was converted into a Shorten file, which was converted into a WavPack file. Finally, the WavPack file was converted into an uncompressed Wave file.

The resulting Wave file was exactly the same size as the original Wave file, and the MD5, SHA1 and CRC-32 hash values matched exactly with the original. This is very strong evidence that each of the lossless formats tested preserves the original audio perfectly.

Note that this process only succeeds because Wave does not support meta tags, it only stores the audio in the track. The conversion process above will change meta tags (that describe the track name, year the track was released, etc), so these are not necessarily preserved exactly between lossless formats. This is because not all audio file formats support the same meta tags. However, as far as audio is concerned, these formats really are lossless.

Bobulous Central → Miscellaneous pages → Lossless audio formats 2006.