Let’s say there’s a new film out in a few weeks that was filmed under the following unusual contractual condition: if the Rotten Tomatoes rating isn’t above 80% after a week or two, the writer and director don’t get paid royalties. Or say a new band signs to a record label, and the label refuses to pay a signing bonus unless their first album gets 5 stars in NME, the Guardian and Kerrang as well as an 85+ on Pitchfork

Loading

“ It’s difficult to overstate how much most large publishers, from EA to Bethesda and beyond, care about that Metacritic average, and how it affects every aspect of game development, marketing and PR.

Loading

“ I’ve been told stories of PR executives working on particular games being directed to specifically target these smaller sites in the hope of raising that average score and covering up a less enthusiastic reception from the big outlets.

Loading

“ There’s a reason that the Metacritic average has somehow ended up on a pedestal: it’s a number, an easily quantifiable value.

Imagine how patently absurd this would be – and then reflect that this is exactly the situation that some game developers find themselves in. Metacritic, the review aggregator that collects scores from major sites, newspapers and magazines and swirls them together with blogs and smaller outlets to create an average number and give you an at-a-glance assessment, has become such a powerful force in the games industry that it doesn’t just impact the nature of games reviews, it directly influences development and marketing, as well as how developers are paid.None of the following is necessarily the fault of Metacritic itself, nor the people who run it. There’s nothing inherently wrong with an aggregate review site for a quick summary of critical opinion, and it’s a useful service for film and game fans despite its limitations. What is wrong is the way that Metacritic averages are used by the games industry to determine how games are made and sold, and the negative effect that they are having on criticism.Metacritic rates movies, TV and music as well as games, but only in our industry is that arbitrary average held in such high regard. It’s difficult to overstate how much most large publishers, from EA to Bethesda and beyond, care about that Metacritic average, and how it affects every aspect of game development, marketing and PR. Ideas – we’re talking creative ideas about what to put in a new game, here - are pitched and green-lit based on how they might affect a projected Metacritic rating. PR people are evaluated on how effectively they’re doing their jobs by looking at the aggregate number they’ve managed to ‘achieve’ for games they’re publicising. said that , “unlike many other entertainment business[es], ratings by Metacritic and others' reviews really can influence the success of a newly-released title… if your ratings go below a certain level, it can really hurt your ability to sell the title, and above a certain level can make a real difference in your success.” Both Telltale and Bioware have been caught messing with the user scores. EA’s Peter Moore, head of EA Sports at the time, set the FIFA 10 team a Metascore target of 90 – and then went on to say that basing success on Metacritic scores is a “slippery slope” For understandable reasons, it’s extremely difficult to get people to go on the record about this stuff, but anybody who’s worked in the games industry at any large publisher or as a journalist has stories. Obsidian’s Chris Avellone (perhaps inadvertently) broke the silence in March this year, revealing in a since-deleted tweet that Obsidian missed out on being paid a bonus by Bethesda for Fallout: New Vegas because it failed to hit a Metacritic average of 85. (Tragically, it currently sits at 84.)This doesn’t apply to all publishers and all contracts, of course. But it shouldn’t apply to any. It also creates a very weird problem for games critics, who suddenly find themselves potentially responsible for a developer’s livelihood – as commenters who feel you have underscored a game will sometimes point out. This absolutely should not be a consideration in the critical process. Any reviewer who ups a score out of sympathy for the developer is not doing their job properly, and is not doing their readers justice.This ties into another problem with judging a game’s success based on an aggregate score: the reviews listed on Metacritic come from sites with varying standards of professionalism. Metacritic’s weighting metric does give more established, respected outlets more clout, but it’s still a broken system. Eager, inexperienced writers from smaller sites have been known to give very high scores knowing that their review will appear near the top of the listings and refer traffic. This is all fine when you’re just using Metacritic as a reviews resource, but when that score determines whether a game gets a sequel, it’s a serious problem.A Metacritic average undermines the whole concept of what a review is supposed to be: an experienced critic’s informed and entertaining opinion. Instead it turns reviews into a crowd-sourced number, an average. You can’t average out opinions. If you adore the new Muse album and your Radiohead-loving friend hates it, that doesn’t make it an average album. And yet this is exactly how Metacritic scores are treated by publishers. It punishes divisive games – and honestly, most interesting things are at least a bit divisive.Limbo’s creator Arnt Jensen once said to me that “you’re never going to create anything interesting by listening to what people think”. If you’ve ever wondered why certain publishers are so conservative when it comes to their big franchises, this is why. When any change that you make to something successful might make that Metacritic average drop a few points and make everybody look to their bosses like they’re not doing their job properly, it becomes very hard to justify the experimentation.If you’ve ever complained about the lack of creativity or innovation in big-budget, mainstream titles, you can apportion some of the blame to Metacritic. The slightest five-point drop in that average score as the result of maybe 10 small-time blog reviewers questioning an adjustment or risky new element in a game can have disastrous consequences for the people who actually made it.Meanwhile, there’s been a recent shift in how games are marketed: even the biggest publishers are now realising that Twitter, Facebook and their own community sites are an extremely good way of communicating directly with their fans, who will often have made up their minds on whether they’ll be buying a game long before review scores are out. All of this makes that average review score less relevant, because it can’t be directly correlated with sales so easily.But it still has too much influence. Even Steam embeds Metascores on game information pages. The reliance on Metacritic in certain hugely important parts of the games industry is bad for everyone: it’s bad for developers, it’s bad for critics, and ultimately that means it’s bad for gamers too. Most of all, though, it’s bad for the long-term future of video games; the reduction of a game’s merit to a crowd-sourced numerical average stands in the way of their being regarded in the same way as other valuable forms of art and entertainment like film and music, which are allowed - indeed expected - to divide opinion.

Keza MacDonald is in charge of IGN's games team in the UK and has been reviewing games professionally for seven years. You can follow her on Twitter and IGN