King of the Hill of Stochastic Probabilities: Why Best of 3 and Best of 5 are Both Influenced by Randomness, But Best of 3 is to a Greater Degree

Intro:

This is both in response and inspired by the recent WinstonLabs post on BO3 vs BO5 - https://www.winstonslab.com/news/2017/06/10/bo5-control-maps-grant-random-advantages-teams-numbers/. The guys over there are great reads but their analysis is flawed this time around due to a misunderstanding of some fundamental laws of probabilities and an oversight of the rules guiding map selection in KotH. This post shows how the result of a BO3 match is actually more determined by randomness than BO5 assuming the two teams do not have equal probabilities to win each map, or put another way, at least one team is better on at least one map than the other team but both teams have the same cumulative probabilities over the set of 3 maps (the same scenario posited by WinstonLabs). There are two separate parts to this post. Part one is an argument for why WinstonLabs' logic is flawed and the conclusion that BO3 is fairer in any given match than BO5 is incorrect due to a misunderstanding of how cumulative probabilities work and the second part is my own analysis for why BO5 is less influenced by randomness than BO3. Reading part one is necessary to establish the logic in part two. Part one uses traditional stats that you may have learned in high school or college and isn't hairy if you're familiar with Cumulative Probabilities and The Law of Large Numbers. Part two uses stochastic statistics to analyze whether BO3 is fairer than BO5 because it is a more accurate mathematical model than bayesian stats for understanding KotH results. Stochastic stats is useful for when the order of events matter, such as in KotH where once you lose 2 or 3 maps depending on BO3 vs BO5, you don't get to see another map. I'll give a quick intro to stochastic stats in part two and try to avoid too much math. This whole post is intentionally more focused in logic than mathematical concepts and number crunching in order to avoid ugliness and confusion.

Part One:

The Rules of King of the Hill:

Let's establish how KotH map selection works. In both BO3 and BO5 you know the first three maps which are played but not the order in which they are played. At the conclusion of the third map, you will have seen all 3 maps. If you lose 2 maps in BO3, you don't see the third map, and if you lose 3 maps in BO5, you don't get to see any additional maps (maps 4 and 5 if you lose 3-0 or just map 5 if you lose 3-1). The random factors in KotH are two-fold: The order of the maps played and which maps are chosen for maps 4 and 5 in KotH. WinstonLabs ignores the importance of not seeing additional maps once the defeat threshold is reached (2 maps on BO3 and 3 maps on BO5) but that oversight is actually not important for part one so just table that point for now, I'll come back to it in part two.

It doesn't matter whether a team is favored 51:49 or 100:0 on a given map to show that with the conditions given: two teams which have asymmetrical odds of winning an at least on map and the aforementioned 2 random KotH factors, order of maps played and map selection on maps 4 & 5 for BO5, randomness does have an influence over the outcome but not in the way WinstonLabs says it does. This post does not in any way seek to determine whether randomness in KotH is a significant problem in Overwatch so the hypothetical odds of how often each team wins a map do not matter so long as their cumulative odds are equal and that one team has higher odds of winning on at least one map than the other team. I will however give some suggestions as to how Overwatch could reduce the impact of randomness in KotH in my final thoughts.

Same Cumulative Odds to Win the 3 Maps in Totality, Differing Odds to win Individual Maps:

Let's get down to a specific example. We have Two Teams. TeamVillage and TeamSanctum. They face each other in the Nepal set. If they play on Village, TeamVillage always wins. If they play on Sanctum, TeamSanctum always wins. If they play on Shrine, it is a 50:50 chance of winning for each team. We can see that their cumulative probabilities are in fact, completely even at .5 and .5, the same conclusion WinstonLabs made although the odds are technically .5 and .5 not 1.5 and 1.5 since cumulative probabilities are averaged and probabilities cannot exceed 1. Either way, the conclusion they made that the teams have equal odds of winning on a set of 3 maps is correct. Each team has a 100% chance of winning one map, a 50% chance of winning another map, and a 0% chance of winning another map. Over a large set of matches, say 1000, we would expect that TeamVillage and TeamSanctum each win about 500 times. Google Law of Large Numbers and Cumulative Probability if you're in disagreement with these prior comments. If you are in disagreement here and claim to understand those concepts, reading the rest of this may be a waste of time.

Both BO5 and BO3 start with the first 3 maps in a random order. Useful for us, since this analysis applies to both game modes. Which map is chosen first is a 1/3 probability, the second map is 1/2, and the third map is 1/1 probability, a given. We all agree that over the long haul, these teams have a 50:50 chance of winning this game mode. Let's look at the selection of maps 4 and 5. Map 4 is again, a 1/3 probability since all 3 maps are in play again, and Map 5 is a 1/2 probability since Map 4 is removed from contention. We again can clearly see that this randomness is fair and that there are equal odds of each team winning the section of maps 4 and 5 since we know that all 3 maps are in consideration for maps 4 and 5 with 3 options that occur at the same frequency; 1/3: Village and Sanctum, Sanctum and Shrine, Shrine and Village. On the totality of the 3 maps, the teams have identical cumulative probabilities of .5 and .5, again their odds of winning each map divided by the number of maps, (1+.5+0)/3 = .5. The critical error made in WinstonLabs' analysis is that the selection criteria for the two discrete sets of Maps 1-3 and 4-5 is identical. It does not matter at all how many maps are played. Fair is fair and the cumulative probabilities of these two teams winning is identical.

Conclusions With Respect to the WinstonLabs Analysis:

So we can clearly see that it doesn't matter if the format is BO1, BO3, BO5, or BO999. The odds of each team winning the set of maps are identical. I'm completely confident that any argument which assumes that two teams that have the same cumulative probabilities (doesn't matter if it's 60:40, 60:40, 30:70 or the 100:0, 0:100, 50:50 in my example) and have different odds of winning a set of maps based on the number of rounds played is incorrect. This is a basic rule of how odds work. WinstonLabs concludes in the example of 60:40, 60:40, 30:70 example that it is best to be the 30:70 team. This is again, a fundamental misunderstanding of how cumulative odds work. WinstonLab's own math is at odds with their conclusion here. I'd refer you to the section with 3 BO5 charts where they show 3 scenarios where the odds of winning are 52:48, 52:48, and 46:54 and somehow conclude it's better to be the team advantaged in the 46:54 scenario. This ignores the fact that again, these cumulative odds are (150:150)/3 which is .5:.5. Completely even since the odds of each of those 3 differing scenarios for maps 4 and 5 happening are equal (1/3). I welcome any discussion on this but I hope so far I've shown that these are basic truths.

Where is the Error in The WinstonLabs Logic:

Well I've given my breakdown of how these numbers work but obviously there has to be a nasty little bug in the logic. WinstonLabs is ignoring a few things. I want to go into the error in their model before going into a more nuanced approach to BO3 vs BO5 using stochastic stats. I believe the model they chose is incorrect but that's for part 2. The major error in the implementation of their model is selectively applying cumulative odds. WinstonLabs fails to acknowledge that all 3 combinations of maps for rounds 4 and 5 are equal. Again. With equal cumulative odds and equal frequencies, we still have .5 to .5 for the odds of winning. If this were BO1 that would be true. It would be true if it were BO999. A gambler would still only wager a dollar to win a dollar. When the set of maps is known as in a BO3 where we're guaranteed to see all 3 maps once (again assuming we see all 3 rounds which is not true but they use this assumption so I'm using it for now), their conclusion of fairness is correct. When we are not guaranteed to see all 3 maps in rounds 4 and 5, they fail to acknowledge that even though we don't know which maps we see, these maps occur with the same frequency. The fact that a team may be better on 2 maps than the other team doesn't matter at all here so long as their cumulative odds are identical. The error either is in ignoring the even odds of map selection in rounds 4 and 5 or not understanding cumulative probabilities.

Part Two:

Glad you're still with me. If you don't agree with part one you will likely not agree with this part but I promise you part one is correct and you can see so for yourself if you check out Cumulative Odds and The Law of Large Numbers.

Stochastic Stats and Glossing Over the Importance of the Order of Maps Played:

Stochastic processes are used to understand stats in situations where the order of events matters. I'm not going to go deep into heavy math here so forgive simplifications if you're a math weenie like me. You're all likely intuitively familiar with stochastic stats even if you've never heard the term. And if you work in finance, science, or computer programming, you'll know the Random Walk (cute picture for reference https://people.duke.edu/~rnau/411rand_files/image008.jpg). This top quora answer is a good explanation of stochastic processes explained much more articulately than I ever could https://www.quora.com/What-is-a-stochastic-process-in-laymans-terms. And if this seems complex, don't worry we're using basic version of stochastic processes imaginable; we're not predicting gas molecule movement. KotH map selection and order is simple. We need stochastic stats for understanding KotH randomness because the order of maps matter in an individual game. Even though TeamVillage and TeamSanctum have identical odds of winning a match, regardless of how many maps are played, order matters in any given match. How much it matters depends on the relative strength of each team on these 3 maps but it does matter.

A Bad Draw in Best of 3:

Let's start with a BO3 game. The first map is Village. So what does that do to each teams odds of winning? Well we know before the first map that each team had an equal shot of winning the round. And TeamVillage always wins Village in our example so they're up 1 to 0. This is where I want to emphasize that the individual odds of each team winning a map do not matter to understand this so don't number crunch. I'm using 100:0, 50:50, and 0:100 to keep this really simple. If the map order is Village then Shrine let's look at what happens. TeamVillage gets a win, and there's a 50% chance TeamSanctum never gets to see their beloved Sanctum map. Their fate is decided by the fate of the draw and a certainty (TeamVillage winning Village) followed by fate and a coin flip. Hardly a fair round for TeamSanctum. Again, if we play Nepal 1,000 times, this doesn't matter. But in any given game it matters a ton.

A Smart Guarantee from Jeff Kaplan and The Law of Large Numbers:

So let's get a little more general. There is a 1 in 3 chance that Sanctum is the last map for a given round in BO3 and there's a chance they can lose without seeing a map they're favored on even though these teams have identical odds of winning the round before map selection starts. But what if this is BO5? Well now we're guaranteed to see all 3 maps at least once since you need to lose 3 times and not 2 to end the match. Now each team is guaranteed to see each map once and since they have equal cumulative odds over this set of 3 maps, this is much fairer indeed. Again, doesn't matter if the odds are 1:0 like I said or 51:49. BO5 guarantees that each team sees each map at least once and BO3 does not.

But what about those final 2 maps? Let's go back briefly to the Law of Large Numbers. Randomness is negated over large samples. And this is where I want to deviate intentionally from my really simple example of teams having 100% odds of winning a given map. I've been using this example so that we focus on the logic behind map selection and order and not distract everyone by crunching made-up odds. In reality, we know teams aren't going to win a given map 100% of the time if two teams are of equal cumulative odds. Now imagine the odds are 60:40, 50:50, and 40:60. Each team has a chance to win each map and is favored, albeit significantly, on some of those maps. Same applies for it being 70:30, 40:60, 40:60. The effects of randomness dissipate with larger samples. Blizzard did something smart here. By guaranteeing that we see each map once in BO5, and no map will ever appear back to back in rounds 4 and 5, they have further limited randomness. In BO3 there is no guarantee that we see all three maps. This alone makes randomness in BO5 have less influence on the outcome. Not to mention that again larger samples further negate the effects of randomness. Even if each map were available to play each round and chosen randomly, BO5 would be less impacted by randomness than BO3. The guarantee that we see all 3 maps in BO5 further reduces the impacts of randomness in BO5. It's no coincidence that BO5 is the default mode for KotH.

Final Thoughts:

First thought is I hope this wasn't longer than it needed to be. If you have questions or arguments against these please respond in the reddit thread where it's posted. I don't want to get into the subjective nature of whether BO3 or BO5 are more fun to watch but I do want to touch on minor things tournament administrators could do to decrease the impacts of randomness (and Blizzard could help them do this much easier than how they would have to do it today by adding some features but that's a separate discussion).

Sports are a good place to look for how to handle "map advantage" AKA "home court" advantage in a series. I only know American leagues well but there is no randomness in which team has home court in a given game. The NBA finals are going on and the order of who has home court is 2-2-1-1-1 meaning that the underdog needs to steal one game on the road to win but cannot possibly be eliminated before each team plays 2 games on their home court. The underdog and the favorite are determined by a set of rules and is done so deterministically. Tournaments could certainly use a model like this where the favorite based on past performance has "home court" but personally I don't love it although it could encourage more aggressive play in the group rounds of tournaments in games that don't have much meaning other than the draw of who each team plays.

In BO3 tournaments I think a simple mechanic can work. The team that loses the first map picks the second. If the first map was one they expected to win, they're in deep trouble regardless and map selection won't help their odds at all since they still need to win both remaining maps. If the first map was one they expected to lose, at least they are guaranteed they get to see their favorite map in the series. Personally I enjoy both the speed of BO3 and the adjustments made in BO5 but the randomness is too impactful in BO3 compared to BO5. I'd love to see the losing team pick the second map. If they lose, it's unquestionable that they didn't get a bad draw since they got to pick their favored map or lost on round one if they're expected to win that map.

In BO5 tournaments randomness in map selection is negated in the first 3 maps since we know we get to see each one once. But what about the final 2 maps? Perhaps the team down 2-1 picks the 4th map but this seems to be a real equalizing factor if the team up 2-1 stole a map they weren't expected to win. It would be fun to watch since we'd get more round 5 series but seems like an unfair "catch up" mechanic like that annoying leader shell in Mario Kart. Another solution would be to keep picking these 2 maps randomly which frankly, is much better than picking a BO3 randomly with respect to the impact of randomness. One alternative which I like would be to pick map 4 randomly. I feel like if you're down 2-1 you shouldn't be getting an advantage by map selection and if you go out 3-1, you can't really blame the maps. Then if it goes to map 5, each team chooses a map to ban and they play on the other one, even if it involves playing the same map 3 times over the course of a BO5. The only random map which could have had an impact on the set is map 4 since we know maps 1-3 and can't be eliminated until we see all 3. Sure you have an advantage if you're favored on 2 maps and the opponent is only favored on 1 but all these design decisions have trade-offs. Randomness is the "fairest" in a BO5 since it can't guarantee the team favored on 2 maps to play on at least one of those if it goes to round 5 so I think either the existing mechanic or the ban mechanic are reasonable.