In 2010, the English pop star M.I.A was riding high. Her first two albums were critically lauded for their experimentation and provocative lyrics. The second of those albums performed surprisingly well commercially, selling well over 500,000 records in the US, and spawning the irresistible anthem “Paper Planes.”

Fans eagerly anticipated the release of her 3rd album. That album, “Kaya”, arrived in July 2010 and mystified many fans and critics -- it received substantially worse reviews than her previous albums, and sold a below-expected 100,000 records. Pitchfork panned the album: “The record is a shambling mess,” wrote the music review site, “devoid of the bangers that characterized Arular and Kala, two of the stronger pop albums of the past decade.”

The data backs up Pitchfork’s assessment. We statistically analyzed the most disappointing albums of the last fifteen years using data from the music review aggregator Metacritic and found that “Kaya” is among the 30 most disappointing albums from that period.

M.I.A. is not alone in the business of not meeting expectations; she shares space on this list with artists like Outkast, Brad Paisley, and My Morning Jacket. In contrast, Kanye West, Beyonce and Daft Punk had albums that were surprisingly great, at least according to Metacritic’s statistics.

In what follows, we highlight these disappointing albums and unexpectedly good albums. We also identify the artists who exhibit the most and least consistency in quality.

The critical reception of Miranda Lambert’s albums have been remarkably positive and consistent.

Following the release of an album, critics descend to scrutinize the recording and award grades and stars. Since 1999, the website Metacritic.com has collected these reviews from notable publications and assigned albums a general score based on critcial reception. They dubbed this the “Metascore.” This is only done for albums that received a substantial number of reviews in major publications (see more about the methodology here). The score is on a 0-100 scale, and the average score is about 71.

In order to understand which albums were the most disappointing or surprisingly good, we collected the Metascore for the 12,000 albums listed on Metacritic.com. We removed live albums, greatest hits collections and remastered versions of older albums.

We also removed all recordings for which we did not have two previous albums in the dataset. Although a first or second album might not meet expectations, we were particularly interested in those situations where an artist had a substantial track record and could thus really surprise his or her fan base with something of different quality. This left us with 3,000 albums released between February 2001 and August 2015.

Although there are are numerous ways to approach defining surprising albums, we chose to compare the score of an album with the average score of the previous two albums by that artist. For an album to be considered surprising, we also ensured that the previous two albums were both 10 points higher or lower than the album being analyzed.

The following chart displays the average of the last two albums against the score of the next album. The points for the ten most disappointing albums are in red and the points for the 11 most surprisingly good albums are in blue (Beyonce was 11th, and of course we had to get her on here). The grey points are for all other albums.

Dan Kopf, Priceonomics; Data: Metacritic

We see from this chart that the Moby album “Hotel” is among the most disappointing albums. Moby’s previous albums had received score of 84 and 61, but “Hotel” was a critical disaster, with a score of 47. The following table displays the twenty most disappointing albums from the last 15 years.

Other notable disappointing albums that did not make the top twenty include Bruce Springsteen’s “High Hopes” (38), and My Morning Jacket’s “Evil Urges” (57).

It’s not all doom and gloom. Sometimes artists surprise us for the better. The next table shows the top twenty most unexpectedly good albums.

The rapper Mos Def’s album, “The Ecstatic”, was the most pleasant surprise in our dataset. His last two albums before “The Ecstatic” had received damning Metascores of 45 and 59. Few were expecting his next album to be a return to form (this author included). Instead, the album received an outstanding score of 81.

Another surprising album came from the 2013 eponymous album of world dominating pop star Beyonce. Her previous albums had received middling Metascores in the 60s and low 70s until “Beyonce” received an impressive 85.

***

Jay-Z’s albums have been of a particularly inconsistent quality according to critics.

In addition to looking at which albums were most surprisingly good or bad, we were curious about which artists are most and least consistent in the quality of their output. This was measured by looking at the nearly 750 artists/bands with 4 or more albums with Metascores. We determined the overall average score for all of these artists and then looked at how much an individual artist’s average album deviated from this score.

This next table shows the top 10 most inconsistent bands/artists. Many of the same bands and artists show up on this list as on the disappointing list, but we also see some new entrants like former Beach Boy Brian Wilson and megastar Jay-Z.

Other notable high variance artists who did not make the top 10 include The Black Eyed Peas (19th), Chris Brown (26th) and Madonna (30th).

In contrast, certain artists receive remarkably consistent critical response to their albums. For example, the band The Mendoza Line released four albums between 2002-2007, and three of those albums received a score of 78 and one of 79. The following table ranks the 10 most consistent artists.

Country star Miranda Lambert takes the cake for being both a critical darling and a consistent artist. All four of her albums on Metacritic have received ratings between 83 and 86. Taylor Swift also appears among the top 25 most consistent artists.

***

Our analysis of surprise albums and consistent artists is most certainly up for debate. You may feel that the Zac Brown Band’s “Jekyll + Hyde," one of the most disappointing albums by our measure, is actually their best record. Some Kanye West fans found “My Beautiful Dark Twisted Fantasy," one of our most unexpectedly good records, to be a weak capitulation to more conventional tastes.

Overall though, these lists give a sense of how artists can surprise us for better or worse, and how difficult it is to maintain the favor of critics.

Our next post takes a look at which dog breeds get adopted most often and which ones don't. To get notified when we post it → join our email list.

This post was written by Dan Kopf; follow him on Twitter here.