I’ve written quite a bit about tournament formats in the past — mostly criticizing events with poor or insane formats. To me this seems somewhat natural: tournament formats dictate so much of the outcome of what should be a competitive event and season; so to implement inefficient, unreasonable or unfair tournament formats is a direct attack on the competitive integrity of Dota 2.

One of the key parts of the issue at hand is that there are multiple types of people impacted by tournaments, each with their own motives:

players / coaches / managers: want formats that allows them to prove their worth; but not so many games that they’re spending ages playing matches that have only tiny impacts on their progression.

tournament organizers: want to keep costs down, but viewership (both live and remote) high. Mostly want to limit the number of days in the expensive arenas/venues to a reasonable amount.

fans: want as many teams playing on-stage, but not spend a huge amount of time watching the event (not super long playoff days, not too long that following the event is tedious).

The biggest concern for me (and essentially the final straw in writing this article) is that teams / managers / players — i.e. the people who actually are incentivized the most towards wanting fair formats — are personally attacked by irrational fans for wanting to improve this fundamental aspect of the tournament circuit, even when it it’s a tournament they’re personally not involved in.

Great public response #1

Great public response #2

So let’s look at the information we have about the DPC so far. After some cancellations and Valve revoking Major status from an event — we have 22 DPC events on the calendar, 13 Minors and 9 Majors. With DAC just finished and Starladder around the corner, the public has format information on 17 of these events.

Similar to my previous tournament format analysis, I’ll associate a similar scoring function for the quality of the format at ordinally ranking the teams (slightly punishing formats which don’t resolve all positions [i.e. the 7th and 8th teams ending as ‘7–8th’] — which is something I didn’t do last time).This looks calculates an error value based on “on average, how frequently did a team ranked X by skill place higher than a team ranked Y (by skill), for all Y > X”. I’ve also changed the error function slightly, so the newer values are just base error values (i.e. a lower value means a ‘better’ format). Additionally, there’s now a second tournament format metric “Top 4 Right %”, which is a percentage of how frequently the (true) top 4 teams going into a tournament end up as the top 4 teams at the end.

I took each format of the DPC, and ran a linearly skill distributed set of teams through it 10⁵ times. I also threw in some base test case formats such as various Single Elimination cases. Each head-to-head was simple, that is to say it did not consider non-transitive rivalries (A > B, B > C, C > A).

Finally, I wanted to model another aspect of a format’s quality — how resilient it is to initially bad seeding. To do this, I modeled a perception value within each simulation for each team which was based on the inter-team rating (team[i] - team[i-1]) multiplied by the normal(0, 1) distribution. This means that a strong team which event organizers might undervalue when it comes to seeding (or weaker teams which event organizers might overvalue) will be handled within those model simulations. In a format with randomized Swiss, this should have no impact; but in a format such as single elimination — it’ll have a moderately large impact.