What is "good performance"?

1) Limiting your time spent dead, or at least achieving reasonably equitable value from your deaths.

2) Exceptional use of your hero's kit to achieve kills and avoid deaths.

3) Contribution to the team's efforts, whether they are towards the map objectives, team fights, or pushes.

How do we improve "poor" performance? With teaching!

You got a +23 PBA for this match! Good job!

+ You contributed to 90% of your team's kills!

+ You helped with the objective 82% of the time!

- This hero has multiple stuns. Try to use them to secure kills, or to escape from fights that you're losing.

Closing Thoughts

Hello there! I'm CriticKitten. You may know me from my more popular segment, 7 Days Later, in which I keep track of how heroes are performing after recent balance patches. Here is a link to my most recent edition of that series. You can expect the next post in that series in the next few days.Generally you've known me for creating more numerically oriented posts. This time, however, I've got a topic to discuss that is admittedly more subjective than most: performance. Specifically, how the current system might not be doing the best job of evaluating player performance, and how it might be improved.While this particular subject may not be among my normal areas of expertise, I think my previous experience in data collection and game development allows me a somewhat more unique perspective on the issue that might help contribute to the ongoing discussion, and I think it's important that we all give as much feedback to Blizzard as possible to help improve upon the foundation they are trying to build. And so I hope you'll bear with me as I provide my own personal stance on the issue. If you agree or disagree with anything I say, as usual, please feel free to tell me so in the comments below, or on Reddit. I look forward to a hearty and thoughtful discussion!I am not proclaiming myself to know more about the design or development of the game than the devs themselves. Only they can determine what can or can't be done. I fully acknowledge that some of the things I'm suggesting may not be possible in the game's current form, and it's entirely possible that some will just never be possible at all. So please, don't take this as me trying to tell the devs how to do their job. I am well aware that they've got more knowledge on this topic than me. This is merely my attempt to help drive the performance-based matchmaking system in a direction that emphasizes better play over better stats. And I sincerely hope that even if the ideas presented here cannot be used in their current form, the developers will be able to use these proposals as inspiration to help further evolve the system in the future.Let's start by getting one thing out of the way. I don't agree with people saying that performance-based matchmaking is a bad idea, or that it should be scrapped. I can understand why Khaldor advocated for it so strongly, in fact, and I totally agree with his justification on the subject. The fact remains that the game needs some means of evaluating a player's individual contributions to a match, besides the acknowledgement of having high stats. That said, it's hard to argue with the evidence we have so far. Multiple streamers have tested the system in a variety of ways and seemingly come to the same conclusion....that the system rewards spamming rather than tactics, that some roles are less favored than others, and that players who stat pad will inevitably be deemed better "performers" than those who are careful and smart. This suggests that the current performance-based system is merely a collection of raw stats that are measured against the raw stats of other players of that level, which is honestly not nearly sufficient to evaluate "good" performance.So, what exactly is "good performance"? What does it look like? As I see it, "good performance" boils down to the following core elements:1) Limiting your time spent dead, or at least achieving reasonably equitable value from your deaths.2) Exceptional use of your hero's kit to achieve kills and avoid deaths.3) Contribution to the team's efforts, whether they are towards the map objectives, team fights, or pushes.The end goal of my proposals here are to achieve a system that evaluates performance based on outcomes, rather than just looking at cumulative stats themselves and determining whether or not they were high enough. Yes, I know these are very generalized concepts, and you may be thinking "well, it's impossible to evaluate those things". But I disagree. I think all of these things can be evaluated in some form or another. Keep in mind that the performance-based matchmaking system has been stated to track "at least 20 different stats" for each hero, which means it's able to look at much more than just the stats on your final stats screen. The game engine seems to have some built-in mechanisms that allow it to track specific stats that a replay file wouldn't otherwise contain, which means there is a way for the game to log stats as the game is going on. I think the performance-based matchmaking system can be greatly enhanced by allowing it to review all of the information and actively cross-reference the data while a match is going on, rather than just a few cumulative end-game stats. I'd like to detail how I believe this would contribute to a better system, using the three core facets of "good performance" I've listed above.The game already tracks both the number of deaths and the time that each player spends dead. The former is shown on your in-game stats page, and the latter is tracked as part of the MVP system. These statistics are good and valuable, but they lack one key element that isn't tracked: the relative "value" of a death. Let me give a few examples:Zagara is split-pushing in the top lane of Cursed Hollow fairly late into the game. The enemy team is aware of her presence and rotates up to gank her, but not before she secures the keep kill. Was her death "worth" the loss of the player for an extended period of time? Well, yes and no. Removing a keep from the map opens up a "win condition" for that team, and if her death timer is finished before the next big objective or before the enemy team can capitalize on the kill, then her death was worthwhile. But if the enemy team can then use Zagara's death as an opportunity to open up their own "win condition" or to take a boss and push for the win, her death was ultimately counterproductive. So this scenario could be either a relatively meaningless kill for the enemy team, or it could be a huge step towards the inevitable win....yet in both cases, it's viewed the same way.Genji dives into the back line of an enemy team during a late game team fight, and his efforts secure two critical kills that allow his team to ultimately push for the win. Problem is, he has to sacrifice himself for those kills, and spends most of the near-60s timer watching his team finish the game. Under the current model, the game merely reads his death as a death, and counts the time spent dead against him. But in reality, his death secured two kills on the enemy team's players and helped create the scenario that let his team push to the core. Yes, he died, but he took a gamble and it paid off big time, and led to positive value that should be acknowledged by the system.There are numerous other examples, of course, but I think these two best highlight my problems with the way that the game currently views death. While your ideal should be to avoid it as much as possible, I'd like the game to recognize scenarios where the players' death was more or less significant than just a kill for the enemy team, and decrease/increase the penalty accordingly. To achieve this goal, I think the game needs to go beyond just looking at a player's time spent dead. The game needs to detect "win condition" events that occurred before and after the player's death. Some examples of this being applied in both positive and negative ways would include:[+] If the player killed a keep prior to their death[+] If the player assisted in capturing a boss mercenary prior to their death[+] If the player contributed to the death of at least one enemy before or after their own death[+] If the player's team gained more experience from the team fight (due to level disadvantage) than the enemy team did[-] If the enemy team kills a keep shortly after the player dies (and before they respawn)[-] If the enemy team captures a boss mercenary shortly after the player dies (and before they respawn)[-] If the enemy team killed more members of the player's team than their team did[-] If the enemy team gained more experience from the team fight (due to level disadvantage) than the player's team did[-] If the player spends a significant portion of their time dead during objective spawns, or dies just prior to an objective spawningThis is not an exhaustive list of all options, of course, but these are several areas that ought to be considered. And of course, some of these conditions would have varying "worth" over time (like a boss cap, which is much more valuable late-game than early-game). Changing things in this way would make death less a factor of time spent missing, and more about a factor of value gained or lost. If the enemy team can't adequately capitalize on a player's death, or if the player's death created a significant amount of value for their team that outweighed their death, then that death should have less of an impact on their overall performance rating on that hero. While you don't want to encourage players to be too aggressive, it's important to remember that sometimes the top players make decisions that involve a significant gamble on their part, and when those gambles pay off, they're often that much closer to victory. We need to have a system that encourages players to avoid dying, but more than that, it needs to penalize only "useless" deaths, rather than treating all deaths equally.One of the most important aspects of playing a particular hero well is getting the best use out of your kit. Right now, however, the game doesn't necessarily reward proper use of your kit. A few experiments conducted by prominent streamers and pros indicate that it's possible to get good performance reviews while spamming your skills off cooldown, instead of using them strategically to secure objectives or kills. This can lead to very high raw stats while still actually putting out a poor performance overall. Let's look at some examples of what I'm talking about:A Stukov player spams Lurking Arm while in lane. He is able to pad up some pretty significant siege damage from this, and gets a fairly high total for silence time, but the number of kills that result from his silences are few and far between. Should the player be rewarded merely for having a high amount of silence time, or should the game account for the fact that the ability was being spammed and didn't achieve much kill value? The current system doesn't really account for efficiency in skill usage, only totals. And while totals work well for abilities that are purely damage-oriented, it works rather poorly for skills whose focus is more about providing other effects, like damage reduction or crowd control. In those cases, efficiency matters more than raw numbers.A Valla is fleeing from the enemy team. She drops Rain of Vengeance on top of some of the pursuing enemy team members and stuns them, allowing her to escape. In terms of game statistics, this is merely counted as a stun of X duration, and it might also count as an "escape" (though likely only if her health is low). But it is not granted any additional value beyond that, and the contribution that this single stun gives to the player's performance rating is likely fairly low. Small plays such as this, however, are a big deal in the later stages of the game, where escape and survival can often prevent the enemy from getting closer to their own "win condition". As previously mentioned, players absolutely should be punished for dying late in the game and giving the enemy team a chance to win in their absence....but players who manage to avoid such a fate ought to be rewarded for doing so, too.In both of these scenarios, we see situations where the context of using an ability matters greatly, and yet, the system doesn't appear to have any means of accounting for this context. Yet, much like the previous topic, that context can be found in the play-by-play of the situation itself. The game engine already has the means to detect uses of CC and utility effects, as well as "escapes" to some degree. But now, it needs to learn to evaluate these situations for their specific contexts. Here are some examples of how the player's performance should be adjusted in both positive and negative ways:[+] If the player uses some form of CC which contributes to a kill or retreat from the targeted enemy[+] If the player uses some form of CC which leads to their escape or to the escape of an allied player[+] If the player uses an ability that displaces themselves, their team mate, or the enemy, which leads to their escape or to the escape of an allied player[+] If the player casts a damage mitigation effect (armor, Protected, etc) on themselves or an allied player that provides that player with the means to survive an otherwise lethal situation[-] If the player spams CC abilities on enemy heroes without securing the kill, fleeing from the fight, or forcing the targeted enemy to retreat[-] If the player spams displacement abilities without securing a kill, fleeing from the fight, or forcing the targeted enemy to retreat[-] If the player spams damage mitigation effects on themselves or an allied player that don't contribute any amount of damage mitigation at allSome of these may sound fairly obvious, but the system doesn't seem to account for them at present, which is why I felt it important to mention them. And remember, many of these would also have varying "worth" over time (ex: using CC to secure a kill late-game is worth more than a kill early-game, and so is an escape). Providing the allied player with a higher overall performance score based on the efficiency of their abilities allows the system to properly reward the outcomes of their actions, rather than focusing on achieving certain "levels" of stats via padding. This is not to say that some raw stats aren't worth measuring, of course. It's still important to keep track of a player's damage figures, their mercenary camp captures, etc. But the system needs to also acknowledge that in a game as complex as HotS can get, overall raw numbers aren't always going to tell the entire story.While we have been given assurances that the system takes a large number of factors into account, the amount of influence that a particular map has on this system is somewhat questionable. This is also true of a player's individual contributions to the overall team effort. For example, if a player is responsible for helping in knocking over multiple keeps/forts (either directly or by grabbing an objective), having a lower damage stat shouldn't hurt them much, if at all. This is, of course, the hardest thing to keep tabs on, because each map handles its objectives differently and because of the differences in heroes that encourage you to "go it alone" and not participate as much in team fights. But the way I see it, every hero should be contributing in some way to the overall team effort, even if they're not always with the rest of their team at all times. Let's look at some scenarios which showcase the challenge of properly detecting a "contribution" to the team effort:A lone Gazlowe spends the majority of the early-game pushing a single lane. He misses out on a lot of the early kills, but he avoids death and destroys several structures that gets the team ahead in experience. He then switches it up in the mid/late-game and pushes with the rest of the team to win the match. In the current system, Gazlowe would be penalized for his lower kill contribution and (likely) lower team damage. Yet his early-game contributions laid the foundation for his team's eventual victory. This is a problem because the game is still treating all kills as "equal", even though an early-game kill contributes less to the team effort than knocking over the fort in a lane. Even if the early game is a total bloodbath of kills back and forth, that fort ultimately gets you closer to the true goal of reaching the core.Dehaka's team is losing due to a strong push by the enemy team. He begins rotating between lanes to push the minion waves forward and keep them away from his keeps. This ultimately means he might not be around for every skirmish, but it's also a key facet of playing Dehaka well to make sure that you're using his global presence to keep your own territory safe. How much should that Dehaka be penalized for his lack of presence in fights, especially if he's the only thing keeping his team's keeps from going down? Under the system we have now, the penalty is likely pretty stiff, since Dehaka is considered a "bruiser" and thus is expected to have fairly high hero damage.Raynor is farming lanes and attacking any hero that comes near, even securing some kills on his own. He ends up with fairly high hero damage and siege, and likely a fairly high XP rating to boot, which would reward him well as an assassin under the current system. However, he never contributes to the team effort as far as objectives go. Right now, he'll be treated as if he played the character correctly, even though many of us like to refer to players like this as "Jimmy Laner", that is, a guy who pads his stats fairly well but isn't actually helping his team where it counts. This situation is the most problematic because it's effectively the opposite of Scenario 5: this is a player who probably shouldn't be off laning all game, but is presently treated as a "good" player.As the above scenarios show, there are a lot of problems with figuring out how a player "contributed" to the team. In some cases, it's valuable and smart for the player to be away from his team, pushing down lanes....and in others, it's clearly detrimental. So how do you determine when the team mate is just helping, and when they're stat-padding? Unfortunately, there's no really obvious or easy answer to that question, but I think that by moving the system away from just reporting raw stats and is instead looking at situations and outcomes, we can get a better picture of when a player is trying to help their team and when the player is just padding their own stats. Here are some ways that a player's performance rating ought to be adjusted, both in the positive and negative directions:[+] If the player has a high level of participation in the map objectives and/or is near their team while objectives are available[+] If the player spends the time during objective spawns to successfully eliminate enemy structures (such as tower walls or forts) AND their team still has a reasonably strong performance on the objective[+] If the player is defeating minion waves in the later stages of the game in lanes where a fort or keep is down[-] If the player has a low level of participation in the map objectives and/or is unreasonably far from their team while objectives are available[-] If the player spends the time during objective spawns trying, and failing, to eliminate any enemy structures, or just spends the time in lane killing minions in lanes where a push is not necessaryAs I said, this is the most difficult scenario to evaluate by far, because there are so many ways to get value for your team. I think this area, more than any other, needs to bedependent on the specific hero at play, as well as the game timer. For example, our Gazlowe friend from Scenario 5 might receive reduced penalties in the early-game from not participating in map objectives, but that penalty increases in the late-game. Our Dehaka player, being on a global hero, would get more bonus points from clearing minion waves in the later stages of a game in lanes where the fort/keep is down than another hero who lacks that global ability, like a Raynor.It's all well and good to push for outcome-driven performance reviews, of course, but all of this effort would be meaningless if we don't talk about one of the biggest areas that the current system does poorly: teaching players what they did wrong. Right now, the existing system for PBMM is a number. It provides absolutely no transparency for the players. They have no way of knowing what they did right or wrong, and are forced to guess. This is what led to so many of the experiments from streamers as they tried to test what the system is looking for. But it goes beyond just the system's not telling players about what does or doesn't work on that hero. It comes down to one big issue: the game isn't trying to teach you how to get better. It's merely reporting how it feels you did in comparison to the stats of other players, without any context to explain its findings, so to many players it just feels like points being taken away or awarded without any justification.The game already has a way to encourage players for things they've done well with those little starred comments you sometimes get at the end of your games. But right now, these only provide encouragement based on raw stats....which is the equivalent of getting this at the end of a match:Obviously, they need to be a little more helpful than that. My ideal scenario is one in which those little messages are added to the end of every single Hero League or Team League match, and they provide both positive and negative measures that help a player understand why they got the performance-based adjustment that they did. For example, an ETC player might get a message like this:Having a system which provides the player with clear feedback based on the outcome of their match will help that player learn what they're doing right and what they should avoid doing in the future.But it's not entirely the system's fault, of course. Another big problem is that the community has made some amazing educational resources that aren't being adequately used. I had recently been watching one of Trikslyr's streams and he made mention of the fact that he has noticed that the community puts out a lot of great material....and it never gets read or watched by the folks who need it most. This is another thing that I think the game needs to work on improving: making community content creators more visible to the general population of the game. Blizzard has been working on creating content that helps players learn from pros about how to play certain heroes, and that is a good first step, but there's so much amazing community content that could also be a positive resource for the players.Since the resources for creating more expansive tutorials are likely pretty limited, this would be a great way to allow players to learn the game without having to search long and hard to find it. This is something that Steam does fairly well by allowing players to look up Steam-native guides (though admittedly, the quality of those guides tends to vary dramatically). Perhaps Blizzard could do something similar by allowing players to access a "browser" of sorts that contains popular educational guides and videos submitted by our own community. This is something they would likely have to coordinate with the team in charge of the Blizzard launcher itself, and it is obviously not something that could be created quickly. This is more of a long-term goal than anything else. But I believe it is perhaps the best way to make it easier for players who truly want to learn have easier access to the great resources our community has to offer.I strongly believe that a performance-based system is a good idea for HotS, but its current form doesn't appear to be evaluating performance in a way that actually makes sure the player is doing things correctly. Instead of looking only at raw cumulative stats, the system needs to focus on an outcome-oriented approach, and reward players who achieve positive outcomes for their team (and, of course, punish the players who don't). In addition, the system needs to do a better job at being transparent about what a player is doing right or wrong. As a more long-term goal, the game should look into finding ways to grant players greater access to community-made educational content, much like other game platforms do.Do you feel that my proposals have some merit? Or do you disagree with me entirely? What do you think the PBMM system should or shouldn't do, or do you still feel strongly that we shouldn't have one at all? Please let me know what you think about this subject in the comments section below, or on Reddit. I look forward to a very thought-provoking discussion with all of you!