I have previously written at length about problems with transparency and clarity in Valve’s Dota 2 system so as you might imagine I am thrilled to see they have finally committed to a much more transparent system.

That said, I think there are still some very significant issues that need to be addressed. The reason I say ‘more transparent’ as opposed to just ‘transparent’ is that there are still areas where transparency is lacking. This is the first set of issues I’ll address.

Direct invites to third party events

It’s always been the case that third party events have some fairly arbitrary power over which teams ultimately qualify for Valve events. This is because they get to decide on direct invites without having to justify them to anyone. Naturally, teams who attend more events have a better chance of proving themselves worthy of invites.

Sadly, the new system actually exacerbates this problem. The greatest irony here is that this is a problem that Valve’s own discretion actually helped to combat in the previous system. They used to be able to look for problems like this and consider them when selecting invited teams. But now that the TI invite system is entirely automated, third party organisers have a much more direct influence over who does and doesn’t qualify. Any event organiser can look at the current point standings, calculate which teams need how many points, and based on that information deliberately invite those teams whom they might want to help qualify for TI. For example, we don’t want a situation where Na’Vi and VP are automatically at every Epicenter or LGD and iG are automatically at every ACE sanctioned event.

This is a specific problem based on ulterior motives and collusion but there is also a more general version of it. The simple fact is that attending more events means you can earn more points and earning more points means you’re more likely to make it to TI. So if certain teams are getting direct invites to third party events it is fundamentally necessary for those invites to themselves be based on transparent criteria. Otherwise the TI system is still not transparent, albeit one step removed. You might suggest that Major invites should be based on performances at Minors. But consider then how Minor invites will be determined and the same issue arises.

Solution: To solve this problem Valve need to insist on certain criteria from third party organisers when determining direct invites. So far they’ve said that at least one qualifier per the main six regions is required. But this doesn’t speak towards how the other slots work. The most obvious suggestion is to make invites to Minors and Majors also based on the same point system. But this is a problem because it creates a serious barrier to entry and will lead to the top achieving teams having an unreasonable advantage, likely to stay inside the circuit indefinitely.

What other options are there? I’m actually not sure. Perhaps using external systems that consider even events that aren’t Minors or Majors would help. One example would be Noxville’s rating system. Perhaps a combination of factors would be ideal. This might be a challenging problem to solve but what is very clear is that leaving the decision to the discretion of third party organisers simply recreates the transparency problem at a different place.

The simplest solution, of course, would be to do away with direct invites entirely and make all slots into qualifier slots. But I think there are certain merits in having direct invites, especially if they are based on known criteria, and ultimately this is a debate that is worth an entire article just for itself.

Which third party events?

This is mostly an issue of clarity. We’ve been told there are certain necessary requirements for an event to qualify as being a Minor or a Major. These requirements are based on prize pool and qualifiers. But are those requirements sufficient or there more considerations which have not yet been disclosed? Will Valve continue to exercise some discretion when vetting which events can win bids? It actually seems important that they do in an effort to discourage shady events from continuing to rip off teams and sabotage the game.

So there are good reasons to have more requirements than just prize pool and qualifiers. However, if those requirements are not disclosed, yet another transparency issue arises, which in turn becomes a problem of legitimacy or fairness. Just like we don’t want a barrier to entry for players, we also don’t want a barrier to entry for new events. Event organisers should be awarded bids based on equal grounds to one another. As with the first issue raised, this is a problem both because it opens the possibility of abuse and collusion and at a general level because it ultimately skews the qualification system to certain events or regions (this, in turn, will actually reinforce the problem of events having to discretion to choose direct invites).

Solution: The solution to this is simply to be open about all the criteria for winning bids. Even if one of the criteria is a process of vetting that is not completely transparent, just knowing that it’s part of the process is an improvement because independent parties can always make their own assessments as an accountability mechanism.

Money literally buys power

Qualifying Points will be awarded based on the total prize pool of a tournament, and whether the tournament is a Major or a Minor, with Majors giving more points per prize pool dollar.

This is a quote from Valve’s announcement. While scaling points based on prize pool is a good idea, there should probably be certain restrictions on this process. Obviously it’s great to incentivise competition and it’s likely to ultimately encourage more and more investment in the game. But I also think it might be wise to avoid a situation where event organisers can literally buy influence in the scene. This is an even more serious problem when coupled with the first problem raised in this article.

While not a likely outcome in the near future, a simple reductio illustrates this problem well. Imagine an event puts down so much money that the total points won at that event are worth more than all other events put together. Obviously that’s bad for the ecosystem because other events won’t be taken seriously — see the much discussed TI effect in China. While this kind of extreme might not arise anytime soon, it is not difficult to imagine problems arising based on drastically different prize pools.

Another way to think about this is in terms of our current conception of which events are the most important. I certainly think that a significant aspect of this is which teams are in attendance. Under the new system you could theoretically have a 16 team event with the mandatory 6 qualifier slots and 10 direct invites that all go to teams who aren’t in the top 16 rated teams in the world. If this event had more money than another event whose direct invites were directly based on the same rating system, it would seem bizarre to say that the event with more money but far weaker teams matters more than the event with less money but far stronger teams.

Solution: As suggested at the beginning of this section, the solution to this problem likely lies in setting caps to how many points events can be worth.