d_ijk_stra Profile Joined March 2011 United States 36 Posts #1



I've written a short article on StarCraft 2 balance!

I hope SC2 geeks especially those have science/engineering background would enjoy it...!



http://arxiv.org/abs/1105.0755



The analysis states that T > P, T > Z, and Z > P while the statistical significance is not very strong. Any feedback would be welcome.



Only recently I've found that there has been other statistical researches/discussions on teamliquid. Sorry for not citing them in this version, and I'm planning to update this article with references in the future.



The difference of this article and previous one is that here I tried to take each gamer's ability into account. That is, oGsMC dominating Terran gamers does not necessarily mean that P >> T, since it could be that actually TvP balance is quite good but MC is just too strong. By taking each gamer's ability into account (using what statisticians call 'latent variable'), I think I resolved this problem.



I hope people enjoy it! :D This is my first post on teamliquidI've written a short article on StarCraft 2 balance!I hope SC2 geeks especially those have science/engineering background would enjoy it...!The analysis states that T > P, T > Z, and Z > P while the statistical significance is not very strong. Any feedback would be welcome.Only recently I've found that there has been other statistical researches/discussions on teamliquid. Sorry for not citing them in this version, and I'm planning to update this article with references in the future.The difference of this article and previous one is that here I tried to take each gamer's ability into account. That is, oGsMC dominating Terran gamers does not necessarily mean that P >> T, since it could be that actually TvP balance is quite good but MC is just too strong. By taking each gamer's ability into account (using what statisticians call 'latent variable'), I think I resolved this problem.I hope people enjoy it! :D My Life for IU!

Barca Profile Blog Joined October 2010 United States 418 Posts #2 I have confirmed your results by looking through my match history. - I hate threads that end with "Thoughts?" -

awesomoecalypse Profile Joined August 2010 United States 2235 Posts #3 Very interesting that everyone cries about Protoss being too strong, yet not one statistical analysis backs it up in any way. Thanks very much for posting this, and welcome to TL. He drone drone drone. Me win. - ogsMC

Archas Profile Blog Joined July 2010 United States 6505 Posts #4 Nice first post. Even at a glance, this is very interesting stuff. Good work, and welcome to TeamLiquid. The room is ripe with the stench of bitches!

Malpractice.248 Profile Blog Joined November 2010 United States 731 Posts #5 Wait, you think Z > P, at all levels, or on average?

At top lvl, its quite a diff story...

Zeke50100 Profile Blog Joined February 2010 United States 2220 Posts Last Edited: 2011-05-05 01:05:46 #6



This is pretty cool to look at. Unfortunately, the field is a bit too dynamic and littered with confounding variables that it's hard to pin-point any valid conclusions, but it does show that overall, there's not much evidence to say there's something horrendously wrong. 55% win rate does not mean imbalanced, contrary to popular belief XD A thread with "Statistical Analysis" in the title with an actual statistical analysis within? Impossible!This is pretty cool to look at. Unfortunately, the field is a bit too dynamic and littered with confounding variables that it's hard to pin-point any valid conclusions, but it does show that overall, there's not much evidence to say there's something horrendously wrong. 55% win rate does not mean imbalanced, contrary to popular belief XD

d_ijk_stra Profile Joined March 2011 United States 36 Posts #7



Malpractice/ Yes in my experience I feel like P > Z... But the technique I used takes skill of every gamer into account, so I think possible explanation is that low level Protoss players lose a lot to Zergs, while small number of top level Protoss players are dominating Zerg players. You've probably seen gamers like IMLosira easily dominating GSL code A Protoss gamers... I've been keep reading TL articles, not just posting anyMalpractice/ Yes in my experience I feel like P > Z... But the technique I used takes skill of every gamer into account, so I think possible explanation is that low level Protoss players lose a lot to Zergs, while small number of top level Protoss players are dominating Zerg players. You've probably seen gamers like IMLosira easily dominating GSL code A Protoss gamers... My Life for IU!

HolyArrow Profile Blog Joined August 2010 United States 7104 Posts #8 This is probably one of the most (if not the most) academically rigorous analyses yet, so it'll be interesting to see how people who think P is imba will spin this... I envision many "you're just too academic and not looking at reality" arguments.

d_ijk_stra Profile Joined March 2011 United States 36 Posts #9 Maybe this kind of analysis is more adequate for stabilized BW, but its balance does not interest me anymore :D



Yes, I've seen too many posts which contains 'statistics' in its title with no real serious statistics in it, so I wanted to do one myself Zeke50100/ Yes, I strongly agree with youMaybe this kind of analysis is more adequate for stabilized BW, but its balance does not interest me anymore :DYes, I've seen too many posts which contains 'statistics' in its title with no real serious statistics in it, so I wanted to do one myself My Life for IU!

Malpractice.248 Profile Blog Joined November 2010 United States 731 Posts #10 Well yes, losira was dominating them, but had a harder time vs code S ones, his skill was just code S material :p

dave333 Profile Joined August 2010 United States 915 Posts #11 Very interesting and it is hard to argue against the analysis, but I guess the only hole is still the lack of enough games played to have statistical significance.

setmeal Profile Joined March 2011 162 Posts #12 great thread, do keep them coming. I would love to see more of such analysis in the future!

Dommk Profile Joined May 2010 Australia 4463 Posts #13 That is, oGsMC dominating Terran gamers does not necessarily mean that P >> T, since it could be that actually TvP balance is quite good but MC is just too strong. By taking each gamer's ability into account (using what statisticians call 'latent variable'), I think I resolved this problem.



I think this is what Liquid`Tyler was saying a few months ago, there are enough pro-gamers and play styles right now that we should be focusing on the player and not the race I think this is what Liquid`Tyler was saying a few months ago, there are enough pro-gamers and play styles right now that we should be focusing on the player and not the race

PopcornColonel Profile Joined March 2011 United States 679 Posts #14 Lol cool. I'm looking at Cornell. Zerg delenda est.

mikyaJ Profile Joined April 2011 1833 Posts Last Edited: 2011-05-05 01:19:19 #15 On May 05 2011 09:54 Malpractice.248 wrote:

Wait, you think Z > P, at all levels, or on average?

At top lvl, its quite a diff story...

yea, zergs are winning even more:





jk that's only Korea, Zergs are ONLY winning 51% of the games versus Protoss in the foreign scene:

yea, zergs are winning even more:jk that's only Korea, Zergs are ONLY winning 51% of the games versus Protoss in the foreign scene: MKP||TSL

JerKy Profile Blog Joined January 2011 Korea (South) 2321 Posts #16 That was a really good read, good work! You can type "StarCraft" with just your left hand.

professorjoak Profile Joined July 2008 316 Posts #17 Data set had only about ~620 nonmirror games in it. It would be interesting to use this methodology on the Brood War TSL Season 1 and 2 full ladder replay packs, which have several times more data in them.



I looked into trying a statistical analysis for TSL Season 1 at one point to see if the distribution of build orders on a map had any correlation with win percent. A first glance at the data showed all matchups on any map where I had 100+ games in that specific map and matchup balanced within 52-48. (Which is different than the Korean results in the TLPD which usually split 60-40 or 55-45, though those are based on far fewer games). However, I then realized the data set had many duplicate games from a game between two top ladder players being counted in each player's replay pack and decided it would be too much trouble to properly sort them out so I quit there and didn't take the analysis much further. "The different branches of Arithmetic -- Ambition, Distraction, Uglification, and Derision." --Lewis Carroll

JKira Profile Blog Joined April 2011 Canada 1000 Posts #18 Can't seem to open the PDF, but from what I gather the data taken for this analysis are from major tournaments in the first half of April? I ask because it's not in the OP

zeek0us Profile Joined October 2010 United States 67 Posts #19 LOL SC2 on the arXiv??!?!? Nice. I don't usually peruse the statistics section, but maybe I should.

Mordiford Profile Joined April 2011 4174 Posts Last Edited: 2011-05-05 01:35:59 #20 Yeah, I think it's important to note that a sample size under 1000 games may not be very accurate for something like this where variation does occur based on just players performances and all the shit that influences them.



The graphs posted above by mikyaJ I feel are a little more viable, particularly the foreign tournament one, considering that it's based on a fairly large sample size of over 8000 games. Based on that, it would seem as though the game is fairly balanced in terms of win/loss.



The korean sample is much smaller because of the number of tournaments played, it's basically just a representative of the GSL right now, if I'm not mistaken.

1 2 3 4 5 6 7 Next All