Spoiler

Michael Wetzel, Big Data Engineer

When the sandbox team came to us and asked us if we could somehow measure weapon effectiveness and balance, we were super excited about the idea. The concept of being able to distill weapons and player usage into metrics that we could compare was very compelling from an analysis standpoint, and could potentially provide huge value to our designers. The challenges however, were substantial – trying to come up with a single number or set of numbers that accurately describes a weapon is a very difficult task, as each weapon has its strengths and weaknesses, situational suitability and “places” in the sandbox. Normalizing this data is also challenging – for example, in Warzone we see a huge number of AR and Pistol kills, does this mean they’re the best weapons? Probably not.



Initially we looked at kills per weapon per death as a starting point, but this turned out to be non-ideal. For example, how do you measure weapons that are thrown away after being used? Next on the list was kills/weapon instance, but it turns out there are some potential issues with this metric also. How do you deal with spawned weapons on the map versus picking up weapons from dead players? Also, how do you deal with loadout weapons that are never actually used by the player?



The main metric we landed on was kills per weapon use. This allows us to easily compare weapons no matter where they came from based on whether they were used or not. It's not without complications though, because we have to define use as something that is fair to all weapons. For example, if we only considered use to be shooting the weapon, then weapons that have more deliberate shots (e.g. Railgun) would naturally have an advantage since the usage wouldn't be counted unless they get their shot off. That's why use has a time component – if you hold the weapon as your primary for more than 5 seconds, it counts as being used. In addition, damage dealt per use is also important, especially for weapons like the Plasma Pistol that are great at dealing out damage or nuking shields, but maybe not great at racking up kills.



Along with this, for each of these data points, we capture information about the game mode, the map, the killer’s and victim’s respective skill levels, distances, etc – this allows us to do cool drill-downs on the data to understand which weapons are more favored by higher skill players, which weapons are most effective at which ranges, and on which maps, and if they’re overpowered on a particular map.



We can of course find cases where these details are not 100% representative of effectiveness in particular instances, but after many iterations, we’re pretty happy with how it allows us to interpret the data, especially when we look at it over the entire dataset of the billions of kills that have happened in the game.