The Problems with PGR: A 5 Part Journey. Likes/RTs very appreciated.

A proper global ranking system that accounts for all regions is essential for a competitive games health as well as for legitimizing a game in the eyes of outsiders. One of the essential pieces of a ranking such as this one is that the general public has an understanding of the rankings and its methodology at least on a surface level to add a layer of trust. The Panda Global rankings does not stand up to the industry standard set by other rankings in other major sports and esports such as golf and tennis for many reasons including the (aforementioned) trust. In this I will be dissecting what makes the PGR so dicey, and why every top player I’ve talked to personally has had and continues to have many gripes with it.



PART 1: The Mystery



The one major thing that has rubbed me the wrong way time and time again with the PGR is the commercialism of the rankings and the consequences that come with it. The PGR is released in 6-month seasons with each 10-person section being released individually to maximize clicks. The problem that arises from a system like this is that the commercialization has made it to where the rankings and its algorithms must be kept a complete secret so nobody can find out the rankings before they are officially released. One of the strengths of an algorithmic ranking system over a panel-based system (such as the one used with the Melee Top 100) is that it supposedly eliminates the human bias and gives a sense of trust and objectivity. When you have an algorithm that is independently owned, created, and taken care of by a group of one company’s employees, that bias is now re-implemented just by a machine instead of a person. There is no middleman in this situation or anyone that can check their work and see why any of the inner workings are, well, working. If the idea of an algorithm being biased doesn’t make sense to you, I recommend googling “why algorithms can be biased” and reading through a couple of the articles that pop up, a lot of them are pretty eye opening. I don’t think algorithms are inherently flawed in any way, the problem with algorithms comes from when nobody has access to it to help adjust it.



PART 2: The Industry Standard



The two rankings that I will be using for comparison in this section are the “Official World Golf Ranking” (http://www.owgr.com/ranking) and the “Association of Tennis Professionals Men’s Tennis Rankings” (https://www.espn.com/tennis/rankings). One thing that you’ll notice very quickly by checking both rankings is the points. You can see how many points someone has at any given time and more importantly the change from last week. That’s right, WEEK. This is the standard that top sports and other competitive games use to track the skill levels of their players. A public and digestible ranking that is updated on the week’s events (which are assigned point values based on depth of competition just like in Smash!) that has a view-able algorithm. For a while the OWGR was a really flawed system, but then Sony came along and sponsored the system and demanded that because their label was on it that it has some quality attached to it as well. This didn’t immediately fix the rankings, but because of this decision by Sony the system was made public and every year it was adjusted based on other's recommendations. Outliers would pop up on the rankings and people would see the weird parts and be able to find the cause and adjust the algorithm accordingly. These adjustments kept the integrity of the rankings while also fixing the quirks that would cause undeserving players to skyrocket. We’ve already had our big company come along to try to help our rankings out. Unfortunately, unlike Sony, the seal of quality wasn’t pushed by Panda Global, so we are left scratching our heads and having the whole community being left in the dark to blindly trust the PGR simply because they have good marketing. We can do better than that.



PART 3: Panda “Global” Rankings and the Goofy Algorithm



Plain and simple, the PGR is not a Global Ranking. PGR is a ranking that heavily favors the US (and Japan to an extent) and whichever poor sap that's willing to drop thousands of dollars *each trip* to even have a chance of making it on. A global ranking is important to say the least, people make careers off of Smash and your position globally can lead to sponsorship opportunities and can facilitate someone’s livelihood. Making people outside of the US spend their life savings just to jump-start that is just entirely cruel. Not to mention the fact that they could potentially not even make it on due to wacky algorithm things such as letting people with 1 win onto PGR because it was at an S tier or because they beat MKLeo’s Duck Hunt in round robin invitational pools. The PGR has systems in place to attempt to remedy this problem such as the attendance requirements (requires 170 people for a C-tier outside of US, 200 inside US), but the problem is not attendance. Europe can have C tiers, B tiers, and A tiers as much as they want, what matters is wins. Countless times people have made it on PGR through just having one good run where they get a top 10 win. When you’re a European smasher, you don’t even have a chance. There is not a feasible way for European players to be on PGR in the first place, which spirals and makes it to where no European players can get PGR wins and thus can’t make PGR. They are literally screwed from the beginning and there is not a way to get around it with the current algorithm. Meanwhile there are people in Japan and Tristate with all losing set counts but because they have so many chances to farm wins on their neighborhood PGR player, or they get a one-off win on Tweek’s Ridley (sorry Frozen), they get to join this exclusive “Global” ranking.



PART 4: Conclusion



I used to be a very big proponent of the panel system used by the Melee top 100. I now see that there are both good and bad parts to both the algorithm and the panel based system, but because there are not enough informed/reliable statisticians to look at the data for Ultimate yet, I think sticking to an algorithm is the right choice. But that doesn't mean that we should stick to what we’re working with right now. I think for starters switching to a system that is updated on a regular (weekly) basis and thus has more care put into it would be a very responsible and progressive thing to do for the health of the game competitively. It also allows for people to wisely choose the events that they go to. People often suffer from burnout or on the other hand don’t attend enough simply because they don’t know where they stand globally or if they need to go to more. Having a consistently updated ranking can remedy a lot of the problems competitors have with the ranking on its own. This change would also allow the algorithm to be publicized to allow constructive input to be given to improve the ranking. This isn’t something that needs testing or needs to be thought about at length either, this exact system has been implemented several times with several other sports to great success and extremely minimal backlash, even from the top competitors.



PART 5: The Multiple Cherries on Top



This is essentially the tangent section of this twitlonger/stuff I didn’t want to give their own part. One of the first things that bugged me was that I asked on twitter for people’s problems with PGR and the amount of variation in responses was genuinely disturbing. Nobody seems to have literally any idea what the fuck is going on with this ranking. There were people complaining about it valuing attendance too much and people complaining about it valuing attendance too little etc, there is so minimal information given out that people are left guessing what's even being weighted. One thing however that was completely consistent was the complaint that there isn’t enough transparency, and I agree wholeheartedly. One other thing that I think needs its own little segment is how the PGStats team handles backlash. I think it’s pretty unprofessional generally that almost all criticism that can’t be answered about the PGR is met with “you don’t know what you’re talking about” or something similar, almost all of the defense can be boiled down to the team gloating that they’re the ones with experience and that you don’t know shit, even if the criticism is generally very valid. You don’t need a degree in stats to see that certain areas of competition being weighted in very lopsided ways is unhealthy and should be adjusted, and I think that handling backlash by taking the high ground position is childish. Not to mention the fact that they take any well-constructed criticism and brush it off/sweep it under the rug and just point and laugh in private because they know giving attention to good criticism will cause their public reputation to dwindle. Just to top it all off, here’s a list of some other random annoying quirks with the PGR:



- No off season. Burnout is a major problem with top players, especially ones in the cutthroat top 20 area that feel pressured to go to essentially every event and put up at every single one unless you’re one of the lucky ones that had 2 straight good performances and would be at their highest if they just don’t attend anything for the rest of the season. The defense for this is that it would screw over tournament organizers, but if the off-season is announced at the very start of the season/if the PGR always ran on an exact schedule then tournament organizers have ample time to plan their events to not be in the off-season.



- The fact that only the top 50 of the ranking is ordered despite the system being algorithmic so it should be able to go forever but it just doesn’t for some reason???



- Nauseatingly high saturation of highly weighted events (S-Tiers)



- Top 100 players/Area 51 players do not contribute anything to a tournaments ranking. There were multiple events in this past Fall 2019 PGR season that didn’t count despite having high depth of talent because everyone in attendance was just outside of top 50. GTX 2019 didn’t even count for PGR at all despite having Cosmos, Lui$, BestNess, Prodigy and myself in attendance.



tl;dr – fuck pgr. read the whole thing please, i put a lot of time into this and it’s pretty informative. it's worth the read :)

Reply · Report Post