Blending Objectivity With Subjectivity in Reviews

As within all industries, video games are a fascinating topic of discussion precisely because they are new. There are always new problems, new methodologies, and new ways to look at how we interact and relate to the medium. That beings said, I think there is a serious problem with review scores in the video game industry.

Due to its objective standard of criticism and homogenized review scales, we have created an environment in which technological achievement is valued over creative merit. Considering this, it makes sense that the argument for games as art is so easily laughed off by proponents of other artistic mediums. In order for the industry we enjoy to be held in equal esteem, we have to transition towards a more subjective system.

Of course, this will bring about its own problems, for the industry as a whole must change or the changes will be misrepresented. Why are the review standards for games so different than what is commonplace for television, music, books, and film? How are tastes accounted for with objectivity? And what effect does our current rating systems have on the diversity and creativity of game design?

Meta-Critic and the Like Have a Backwards System

It is my position that the biggest problem video games face is directly connected to the meta averages of review scores. I would like to take this opportunity to discuss a meta-aggregate for another medium and see how it differs from that which is used in video games. The go-to source for movies is undoubtedly Rotten Tomatoes. It takes a rather binary approach to the way it represents data: the reviewer liked the movie or the reviewer did not like the movie. As such, a review average of 50 percent does not necessarily equate to a failing grade; rather, it identifies the movie as being divisive. Half of the critics had a positive reaction, while the other half reacted negatively.

This, I feel, encourages subjectivity in reviews. It also encourages readers to at least skim through the review highlights. This is in direct contrast to the services commonly used for video games. Both Metacritic and Game Rankings convert their review score sources into a 100-point scale format and then find the mean of the aggregated scores. While a 50 percent on Rotten Tomatoes suggests a divisive film, a 50 percent on Metacritic warns of a nigh unplayable game. This is the tyranny of objectivity, an ideal that is blind to subjective strengths.

In addition to the perception of the meta-aggregate score, there is another problem with the Metacritic model. Scores are forced into a 100-point scale and the intent of the reviewer is often lost in that translation. Consider the common point scales with games: the top choices seem to be a 100-point, 20-point, or 10-point scale, all of which cleanly tie into an accompanying percentage system. A 90 is 90 percent, a 9.5 is 95 percent, and an 8 is 80 percent.

The majority have learned to look at percentages as letter grades, due to the role it plays in education. For example, a 65 percent is a D. Many review sites have transitioned towards a 5-point scale, which is more common with other artistic mediums, though. Different connections can be made to letter grades in that type of scales, so if a site gives a 3 out of 5, we could perceive that as a C. However, when that score is translated into a 100-point system, it becomes a 60 percent, which could be seen instead as a D-. Consider a Yes/No system. Would every “yes” be a perfect score? Would every “no” be a failing grade? As you can see, those outlier scores can give a very different message, once shoehorned into a 100-point scale.

The 5 Point Scale or the 10 Point Scale: Which is better?

Let us dive a bit deeper into the psychology of point systems now. Some would see the above information and lay the fault at the feet of the outliers. I, on the other hand, would not. I feel that 5-point and below scales are more decisive, lead to a more honest and subjective opinion, and encourage the use of the entire scale. There has been a joke around message boards and other gaming communities that a 10-point scale starts at 7 and ends at 10, as if the first seven points are sheerly based on technical competency.

It is my opinion that 10-point and above scales are suited for technology rather than art. These scales tend to encourage objectivity, which often comes at the expense of honoring creativity and fun. I feel that, by grading in such precise detail, we lose sight of the unique strengths of a title and instead honor homogeneous design and bloated production budgets. It is important to understand that numbers have a sort of psychology to them: they can validate our choices and affect our perception of our own interests. They can encourage us to disregard our personal tastes in favor of a general consensus.

While discussing the problems with review scores, I have put a lot of my attention towards alternatives in general. The primary purpose of a video game review is to assist the reader in making a pricey choice and it is for this reason that I dislike objective reviews. Everyone has their own tastes,and the goal of recommending a title should be in addressing these tastes.

Here at Nintendo Enthusiast, a compromise was reached in which every reviewed game receives three scores: a High score to discuss the game’s unique strengths, a Low score to discuss the title’s objective quality, and a Final score that represents the reviewer’s subjective opinion. Other solutions have included Yes/No rating systems, which simplify the process into a basic recommendation, and a Buy/Rent/Pass that adds the option of a soft recommendation. Some reviewers have simply decided to let their words speak for themselves and won\’t give a score at all.

Personally, I have imagined a solution in which a Yes/No or Buy/Rent/Pass rating system is subsidized by trait-identifying symbols that draw extra attention to the game’s unique strengths, weaknesses, and points of divisive contention. Reviewers should be able to express their views in the way that best suits them. Meta averages strip a bit of the reviewer’s voice away, forcing their opinion into an unfamiliar shape in order to reach a level of conformity that the reviewer was not necessarily interested in succumbing to.

Two Big Thumbs Down

When Siskel and Ebert introduced their thumbs-up/thumbs-down rating system for movies, it was embraced for the way it encouraged debate and conversation about films. In the gaming industry, we unfortunately tend to embrace homogenization and a hive mindset over unique individualistic opinions. We are encouraged to avoid debate and to accept a singular vision to represent us all.

Resisting objectivity carries problems as well. You cannot do it alone without running the risk of devaluing your own opinion. Destructoid has recently tackled the problem of a top-heavy review scale by changing their rating policy to include the whole scale, despite the standards of others. While I agree with the sentiment, the problem is that they are alone in their stance. Their score will be averaged in with the others and that will create an inaccurate perception.

The only way to bring back the accuracy of the score is to have a universal review scale. Though I would argue that it is not only impossible to achieve, but that the real problem has always been in an averaged score, anyways. Metacritic and Game Rankings would better serve the voice of the reviewers they source by adopting a Rotten Tomatoes-esque approval rating instead.

That leads me to my primary point. Reviews are not absolute; they are merely a tool. Many of my favorite movies did not review well and the same goes for games. Often, the divisive games are the best ones, because they have a bigger impact on the player. Consider the scenario in which two games stand before you: one you will either love or hate and the other you are sure to find competent, though likely unmemorable. In this scenario, avoiding the divisive title might result in missing the superior game experience.

If a large group of critics rated a game, with half giving it top honors and half dismissing it entirely, the games overall score would be a 50 percent. On the other hand, if those same critics universally disliked another game with their scores equaling out at around 65 percent, the mediocre title would be publicly regarded as a superior experience. In actuality, the mediocre game simply had broader appeal.

In that scenario, many gamers would miss the opportunity to play a game they might love to instead play a game they don’t care about it. That would be deeply unfortunate. I hope that my fellow gamers identify review scores as but one of many tools to aid in the decision-making process and that they might also weigh developer precedence, personal intuition, and comparative tastes when making their choice.

In the end, nothing pleases everyone and there is no absolute opinion that we all must subscribe to. This does not only apply to our unique and individual tastes in games, but also to how we prefer our reviews to be presented. For that matter, it applies to everything in life. There is no absolute. There is no singular view. We should strive to push aside the hive mind and embrace our individualistic interests.