Lightwip Profile Blog Joined April 2010 United States 5497 Posts Last Edited: 2012-05-17 22:59:38 #1





You will never find a more wretched hive of scum and villainy.



In this post, I will give and explain my findings on this issue, and attempt to show that Terran has an upper hand in both general play, and especially in the creation of bonjwa players. I won't deny that I'm pretty biased on this subject, but I will try to avoid bias and justify my position through facts and numbers, not bias. It's as easy to say "tank imba, vulture imba" as it is to say "zealot imba, ultralisk imba" without any proof. In truth, I believe that no one unit makes Terran superior, but simply the presence of many options and outs in any given situation. However, let's consider facts rather than opinions. For those of you familiar with statistics, this should be pretty straightforward. For those of you who aren't, I'll explain as well as I can.



My analysis will be based off the aggregate scores of all pro games played from the start of 2002 until present day that are listed in the TLPD. A few notes:

1. There can be no complaints of a lucky season or an outlier player. The Six Dragons era is statistically insignificant. Swarm Season is statistically insignificant. Flash, Boxer, Oov, and Nada are also all statistically insignificant. Maps are also statistically insignificant as an aggregate. All these factors balance out into a value that would be very hard to refute.

2. Over ANY 3-4 year period, these numbers are about equal. Savior's innovations, 's innovations, etc. are all insignificant over time because all the other races are, in the long term, able to compensate for these differences. These statistics are not a relic of a pre-Savior past.

3. 2002 is the logical starting point because it is after the release of patch 1.08, the last significant balance patch. It would make sense to start as early as possible, but not to evaluate a game with a different set of rules.

4. It would be neither viable nor useful to look at non-pro games. At any level other than pro, the balance is irrelevant because players simply aren't good enough. If you're not a pro, you pretty much lose only because the opponent played better. Balance is more significant at a higher level, in general (the same rule applies for chess, where white is imba).

5. "If terran is imba, why don't terrans win EVERYTHING?" Because terran is only slightly imbalanced. But as I will demonstrate, a little is enough.

6. It really doesn't matter whether the league victories are concentrated in a single player or spread around many because it's important to realize that if one player wins, then every other player cannot. It only makes sense that better>worse and will win more often than not.

7. Semifinalists and silver winners are completely irrelevant. There are dozens of mediocre and outright bad players who have made semifinals and even finals. Yet you'll be hard-pressed to justify that ANY of the starleague winners certainly didn't deserve to win. There are a few to argue, but even ones like July or Casy aren't even certainly unworthy. Too many non-winners who got close are, though.



Let's start by looking at the MSL and OSL winrates by race.

MSL

Terran: 12

Zerg: 10

Protoss: 4

Total: 26



OSL

Terran: 14

Zerg: 10

Protoss: 9

Total: 33

There really isn't much difference between the MSL and the OSL that would affect this experiment, so we can merge the two and set the winrates in terms of percentages.



MSL+OSL

Terran: 26/59 = 44.1%

Zerg: 20/59 = 33.9%

Protoss: 13/59 = 22%

There's also no need to differentiate between Zerg and Protoss in this test, so we can simply make this in terms of Terran vs. non-Terran.



Terran: 26/59 = 44.1%

Non-Terran: 33/59 = 55.9%

And now, we're ready to conduct a test.



For anyone familiar with statistics, one helpful tool is to test a statistic. It involves an assumed, null, value for a statistic, and a test to see whether or not a statistic of a sample(of a population) obtained is likely to appear by chance if the null is correct. In this case, the sample is all leagues to date and the population is all leagues played and unplayed. Since we do not know the standard deviation(spread) of the population(because it is impossible to acquire in this situation), we will use a t-test with all leagues played as the sample.



Our hypotheses are:

H0: μ=.33 (null hypothesis: the population mean is .33, or a fair 1/3 chance for Terran to win)

Ha: μ>.33 (alternative hypothesis: the population mean is larger than .33, so Terran has a larger than even chance of league victory).



The easiest way to conduct this test is to create a table with the values. It would be long and pointless to list, but it consists of 26 1's to indicate 26 Terran league wins, and 33 0's to indicate Terran losses in leagues. So we run the test:

n(number of sample points): 59

SE (standard error, t-test spread): .501

T value (test statistic measures distance from mean): 1.698

p value: .0475 or 4.75%

Sample Mean: .441 or 44.1%(this was calculated earlier)



Now, most of these values are pretty inconsequential, and are only listed for the purpose of noting statistics. The important thing here is the p value, which is the chance that such a sample mean would appear in a population with a mean as stated in the null hypothesis. As a rule of thumb, if the p value is less than .05 (5%), there is pretty strong evidence against the null hypothesis and in favor of the alternative one. It's not so strong that it's beyond a shadow of a doubt, but this test shows that we have pretty good evidence that Terran does indeed have a higher than fair chance of MSL/OSL victory.



This begs the question: how much higher? Well, let's run a test to create a new model. To do this, we'll need a winrate for all matchups. I added up all the games from 2002 onward (patch 1.08), and here is the result:

TvZ: 6549-5490 (54.40%)

ZvP: 5162-4280 (54.67%)

PvT: 4782-4317 (52.56%)

For anyone involved in BW, this T>Z>P>T trend is not at all surprising. Nor should the ZvP>TvZ>PvT trend be unexpected. At a quick glance, it's obvious that these results are ever so slightly favorable for Terran. If we were to equally weigh the percentages of each matchup (with the mirror being 50%):



Terran: 54.40*(1/3) + 47.44*(1/3) + 50*(1/3) = 50.61%

Zerg: 54.67*(1/3) + 45.6*(1/3) + 50*(1/3) = 50.09%

Protoss: 52.56*(1/3) + 45.33*(1/3) + 50*(1/3) = 49.30%

These are essentially the odds that a player faces in Proleague. So basically, a probable Terran is slightly more likely to win a given game than a probable Zerg or probably Protoss. However, by all means, even over a large period of time this isn't going to make results that are especially telling. Terran will have a higher winrate, but not by much. The imbalance truly comes out in the individual leagues. So let's look at a starleague.





Welcome to the Lightwip Hypothetical Starleague!



The Starleague proper consists of 36 players: 13 Terran, 13 Zerg, and 10 Protoss. Starleagues, unlike Proleague, are not race-balanced; the lower total winrate of Protoss actually hurts the chances of qualifying. If you look at every league in history, Protoss usually qualifies less than Terran and Zerg.



While I could average results from hundreds of simulations to find out the winrate, it would be too difficult to account for all factors and honestly not much more accurate. Therefore, the Lightwip HSL shall have a different set of rules: Victory is winning 16 of 20 games. This is pretty comparable to winning a Starleague proper, even if not exactly the same. By all means, it's a good proxy variable.

Let's calculate the win percentages by race and player count (mirrors are again 50%):

Terran: 54.40*(13/35) + 47.44*(10/35) + 50*(12/35) = 50.9%

Zerg: 54.67*(10/35) + 45.6*(13/35) + 50*(12/35) = 49.7%

Protoss: 52.56*(13/35) + 45.33*(13/35) + 50*(9/35) = 49.2%

This becomes slightly more Terran-favored. By binomial distribution(a situation in which there is a win/lose with a known percentage for each, as here), the chance for a hypothetical player of each race to reach 16 is (really low because 16/20 is an insane record):

Terran: .738%

Zerg: .548%

Protoss: .483%



Scaled,

Terran: 41.7%

Zerg: 31.0%

Protoss: 27.3%

For the most part, these statistics mirror actual SL results, reposted below.



Terran: 26/59 = 44.1%

Zerg: 20/59 = 33.9%

Protoss: 13/59 = 22%

Zerg actually has a slightly higher winrate while Protoss has a smaller one, but predictions are not perfect. It's close enough, at any rate. We could conduct another t-test to see whether 44.1% is far from 41.7%, but I think it's obvious that the new model is a good enough fit for all three races.



Like any statistical analysis, this one is not perfect. I'll outline a few things that ought to be considered below. There are two things that should be considered: bias and confounding variables.



Let's start with bias. Quite simply, there is none. We're not using any data that could be skewed by any form of human tendencies because all these numbers are a fact.



Now as far as confounding variables, there actually is something to consider.

The first is a simple problem: scaling up to 1. We did this to form our model above. This is not necessarily going to ruin anything, but admittedly it's not exactly 100% reliable. While it could generate meaningless data, I think it's not a problem. I could be wrong though, and the model is certainly imperfect.



The second is a bit more tricky: mirror matchups. Zerg and Protoss mirrors often devolve into a coinflip, which allows good players to be defeated by worse players. One consequence of this is that zerg and protoss titles are less concentrated in a few key players, but rather in a bunch of weaker ones. Terran, on the other hand, features numerous key players holding a good number of the titles. As I mentioned before, for one player to win, all others must lose(and the best is most likely to not lose), so simply chalking this up to skilled players is not enough. And on top of that, skilled players are more likely than unskilled players to win against all races, subject to the same conditions. So when a Terran key player advances from a TvT, he'll have more chance than any other Terran of winning his next game against Zerg or Protoss.

Now, this problem would not cause our experiment to incorrectly conclude that Terran has an unfair advantage; on the contrary, if anything it would cause us to underestimate Terran imbalance. But it also brings up an interesting topic: bonjwas. It seems that it is indeed easier for Terran to make bonjwas, simply because not only do Terrans have an advantage inherent in winrates, but they also have a mirror matchup that favors stronger players over weaker players to an extent higher than a coinflip. This certainly does help to explain why Terran spawns bonjwas so readily while Zerg and especially Protoss are hard-pressed to get one out.



I'd like to hear your thoughts and criticisms. Perhaps my logic, analysis, or numbers are somehow wrong. Please, point this out. Brood War is one of the most balanced games ever created. In fact, it may be the most balanced competitive game ever created, an amazing feat given the fact that the three races of Brood War are about as different from each other as they could possibly be. Indeed, the game is very well-balanced, and it's very hard to tell which race is superior when just starting to play. Yet, although it is hard to admit it for many fans of the game, there is one race which is clearly superior to them all. Meet Brood War's favorite child: Terran.In this post, I will give and explain my findings on this issue, and attempt to show that Terran has an upper hand in both general play, and especially in the creation of bonjwa players. I won't deny that I'm pretty biased on this subject, but I will try to avoid bias and justify my position through facts and numbers, not bias. It's as easy to say "tank imba, vulture imba" as it is to say "zealot imba, ultralisk imba" without any proof. In truth, I believe that no one unit makes Terran superior, but simply the presence of many options and outs in any given situation. However, let's consider facts rather than opinions. For those of you familiar with statistics, this should be pretty straightforward. For those of you who aren't, I'll explain as well as I can.My analysis will be based off the aggregate scores of all pro games played from the start of 2002 until present day that are listed in the TLPD. A few notes:1. There can be no complaints of a lucky season or an outlier player. The Six Dragons era is statistically insignificant. Swarm Season is statistically insignificant. Flash, Boxer, Oov, and Nada are also all statistically insignificant. Maps are also statistically insignificant as an aggregate. All these factors balance out into a value that would be very hard to refute.2. Over ANY 3-4 year period, these numbers are about equal. Savior's innovations, Nal_rA 's innovations, etc. are all insignificant over time because all the other races are, in the long term, able to compensate for these differences. These statistics are not a relic of a pre-Savior past.3. 2002 is the logical starting point because it is after the release of patch 1.08, the last significant balance patch. It would make sense to start as early as possible, but not to evaluate a game with a different set of rules.4. It would be neither viable nor useful to look at non-pro games. At any level other than pro, the balance is irrelevant because players simply aren't good enough. If you're not a pro, you pretty much lose only because the opponent played better. Balance is more significant at a higher level, in general (the same rule applies for chess, where white is imba).5. "If terran is imba, why don't terrans win EVERYTHING?" Because terran is only slightly imbalanced. But as I will demonstrate, a little is enough.6. It really doesn't matter whether the league victories are concentrated in a single player or spread around many because it's important to realize that if one player wins, then every other player cannot. It only makes sense that better>worse and will win more often than not.7. Semifinalists and silver winners are completely irrelevant. There are dozens of mediocre and outright bad players who have made semifinals and even finals. Yet you'll be hard-pressed to justify that ANY of the starleague winners certainly didn't deserve to win. There are a few to argue, but even ones like July or Casy aren't even certainly unworthy. Too many non-winners who got close are, though.Let's start by looking at the MSL and OSL winrates by race.MSLTerran: 12Zerg: 10Protoss: 4Total: 26OSLTerran: 14Zerg: 10Protoss: 9Total: 33There really isn't much difference between the MSL and the OSL that would affect this experiment, so we can merge the two and set the winrates in terms of percentages.MSL+OSLTerran: 26/59 = 44.1%Zerg: 20/59 = 33.9%Protoss: 13/59 = 22%There's also no need to differentiate between Zerg and Protoss in this test, so we can simply make this in terms of Terran vs. non-Terran.Terran: 26/59 = 44.1%Non-Terran: 33/59 = 55.9%And now, we're ready to conduct a test.For anyone familiar with statistics, one helpful tool is to test a statistic. It involves an assumed, null, value for a statistic, and a test to see whether or not a statistic of a sample(of a population) obtained is likely to appear by chance if the null is correct. In this case, the sample is all leagues to date and the population is all leagues played and unplayed. Since we do not know the standard deviation(spread) of the population(because it is impossible to acquire in this situation), we will use a t-test with all leagues played as the sample.Our hypotheses are:H0: μ=.33 (null hypothesis: the population mean is .33, or a fair 1/3 chance for Terran to win)Ha: μ>.33 (alternative hypothesis: the population mean is larger than .33, so Terran has a larger than even chance of league victory).The easiest way to conduct this test is to create a table with the values. It would be long and pointless to list, but it consists of 26 1's to indicate 26 Terran league wins, and 33 0's to indicate Terran losses in leagues. So we run the test:n(number of sample points): 59SE (standard error, t-test spread): .501T value (test statistic measures distance from mean): 1.698p value: .0475 or 4.75%Sample Mean: .441 or 44.1%(this was calculated earlier)Now, most of these values are pretty inconsequential, and are only listed for the purpose of noting statistics. The important thing here is the p value, which is the chance that such a sample mean would appear in a population with a mean as stated in the null hypothesis. As a rule of thumb, if the p value is less than .05 (5%), there is pretty strong evidence against the null hypothesis and in favor of the alternative one. It's not so strong that it's beyond a shadow of a doubt, but this test shows that we have pretty good evidence that Terran does indeed have a higher than fair chance of MSL/OSL victory.This begs the question: how much higher? Well, let's run a test to create a new model. To do this, we'll need a winrate for all matchups. I added up all the games from 2002 onward (patch 1.08), and here is the result:TvZ: 6549-5490 (54.40%)ZvP: 5162-4280 (54.67%)PvT: 4782-4317 (52.56%)For anyone involved in BW, this T>Z>P>T trend is not at all surprising. Nor should the ZvP>TvZ>PvT trend be unexpected. At a quick glance, it's obvious that these results are ever so slightly favorable for Terran. If we were to equally weigh the percentages of each matchup (with the mirror being 50%):Terran: 54.40*(1/3) + 47.44*(1/3) + 50*(1/3) = 50.61%Zerg: 54.67*(1/3) + 45.6*(1/3) + 50*(1/3) = 50.09%Protoss: 52.56*(1/3) + 45.33*(1/3) + 50*(1/3) = 49.30%These are essentially the odds that a player faces in Proleague. So basically, a probable Terran is slightly more likely to win a given game than a probable Zerg or probably Protoss. However, by all means, even over a large period of time this isn't going to make results that are especially telling. Terran will have a higher winrate, but not by much. The imbalance truly comes out in the individual leagues. So let's look at a starleague.The Starleague proper consists of 36 players: 13 Terran, 13 Zerg, and 10 Protoss. Starleagues, unlike Proleague, are not race-balanced; the lower total winrate of Protoss actually hurts the chances of qualifying. If you look at every league in history, Protoss usually qualifies less than Terran and Zerg.While I could average results from hundreds of simulations to find out the winrate, it would be too difficult to account for all factors and honestly not much more accurate. Therefore, the Lightwip HSL shall have a different set of rules: Victory is winning 16 of 20 games. This is pretty comparable to winning a Starleague proper, even if not exactly the same. By all means, it's a good proxy variable.Let's calculate the win percentages by race and player count (mirrors are again 50%):Terran: 54.40*(13/35) + 47.44*(10/35) + 50*(12/35) = 50.9%Zerg: 54.67*(10/35) + 45.6*(13/35) + 50*(12/35) = 49.7%Protoss: 52.56*(13/35) + 45.33*(13/35) + 50*(9/35) = 49.2%This becomes slightly more Terran-favored. By binomial distribution(a situation in which there is a win/lose with a known percentage for each, as here), the chance for a hypothetical player of each race to reach 16 is (really low because 16/20 is an insane record):Terran: .738%Zerg: .548%Protoss: .483%Scaled,Terran: 41.7%Zerg: 31.0%Protoss: 27.3%For the most part, these statistics mirror actual SL results, reposted below.Terran: 26/59 = 44.1%Zerg: 20/59 = 33.9%Protoss: 13/59 = 22%Zerg actually has a slightly higher winrate while Protoss has a smaller one, but predictions are not perfect. It's close enough, at any rate. We could conduct another t-test to see whether 44.1% is far from 41.7%, but I think it's obvious that the new model is a good enough fit for all three races.Like any statistical analysis, this one is not perfect. I'll outline a few things that ought to be considered below. There are two things that should be considered: bias and confounding variables.Let's start with bias. Quite simply, there is none. We're not using any data that could be skewed by any form of human tendencies because all these numbers are a fact.Now as far as confounding variables, there actually is something to consider.The first is a simple problem: scaling up to 1. We did this to form our model above. This is not necessarily going to ruin anything, but admittedly it's not exactly 100% reliable. While it could generate meaningless data, I think it's not a problem. I could be wrong though, and the model is certainly imperfect.The second is a bit more tricky: mirror matchups. Zerg and Protoss mirrors often devolve into a coinflip, which allows good players to be defeated by worse players. One consequence of this is that zerg and protoss titles are less concentrated in a few key players, but rather in a bunch of weaker ones. Terran, on the other hand, features numerous key players holding a good number of the titles. As I mentioned before, for one player to win, all others must lose(and the best is most likely to not lose), so simply chalking this up to skilled players is not enough. And on top of that, skilled players are more likely than unskilled players to win against all races, subject to the same conditions. So when a Terran key player advances from a TvT, he'll have more chance than any other Terran of winning his next game against Zerg or Protoss.Now, this problem would not cause our experiment to incorrectly conclude that Terran has an unfair advantage; on the contrary, if anything it would cause us to underestimate Terran imbalance. But it also brings up an interesting topic: bonjwas. It seems that it is indeed easier for Terran to make bonjwas, simply because not only do Terrans have an advantage inherent in winrates, but they also have a mirror matchup that favors stronger players over weaker players to an extent higher than a coinflip. This certainly does help to explain why Terran spawns bonjwas so readily while Zerg and especially Protoss are hard-pressed to get one out.I'd like to hear your thoughts and criticisms. Perhaps my logic, analysis, or numbers are somehow wrong. Please, point this out. If you are not Bisu, chances are I hate you.