Preface

This is part of a series (hopefully) where we’ll dive into the data available from the Halo stats API and try to derive useful strategy from it. While the API is great for providing traditional “box score” type stats (basically what you see in the post game carnage report data), we believe its greatest potential strength, and current weakness, is the play-by-play information. From this, you can determine which strategies are effective and which ones are not. However, the API doesn’t contain all the possible endpoints we desire, so we can’t get a full picture of what goes on in each match. This series will try and illuminate the few things that we can analyze with the available data. This is the first entry in the series, and relies on the API endpoints for game winners/losers and power weapon pickups. As always, feel free to comment on this post with your feedback. If you have questions about how something was calculated, or think things should be done differently, we’d love to hear it.

Introduction

Note: in the most recent Halo update, the starting ammo count for both the Sniper Rifle and the Rocket Launcher were reduced, from 12 to 8 for the Sniper Rifle and from 4 to 2 for the Rocket Launcher. The following analysis was conducted using data from before this ammo count change. It’s unclear what impact that change will have on these results, and perhaps the only way to know is to revisit this analysis in a few months using post-update data.

At the beginning of a match on Coliseum, you’ll often find members of each team discussing who will rush rockets and who will rush snipe. The opening race to each of the power weapons dominates all strategy. Some teams will split forces to try and get both, while others will send everyone to one side to maximize their odds of securing that power weapon. But how important really is it to get first touch on the power weapons? And, if you had to choose just one, which one should you rush for at the start of the match?

Well, we’ll get to those results in just a second, but first a spoiler: getting the power weapons first doesn’t really help your odds of winning. But there is a better indicator of win percentage: total time of possession for each weapon. So we’ll also take a look at that metric and see what we can learn from it. Notice how our success metric is win percentage. There are other metrics you could look at to judge success, in terms of a score based stat. For example, in a strongholds match, about how many capture points is picking up rockets worth? This is something that we’d like to look at in a future post. But for now, we’ll only consider the impact of power weapons on winning a match.

Dataset

To get the data needed for this analysis, we looked at almost 1400 matches (the actual number was 1337 PogChamp) played by pros and top amateur players on Coliseum. Roughly half the matches were Capture the Flag and the other half Slayer. All the matches were Custom Games, and we’re pretty sure that only serious scrimmage and Pro League matches are included. We don’t include Arena matches for two reasons: one, most pros don’t take Arena matches seriously, and two, most Arena matches aren’t played with a full team of 4 players communicating. In both cases, the data from Arena matches may not reflect the “true” meta one may expect to see at the competitive level.

Part 1: First Pickup

First, let’s start with the first, as in “does picking up a power weapon first give us a better chance at winning this game?” To calculate that, all you need to know is which team picked up which power weapon first in each game, and which team won that game. If it seems like the team that picks up the Rocket Launcher first wins the game more often than the team that picks up Sniper first, then we’d say it’s better to get Rockets off spawn.

There are concerns with this approach. For one thing, the average match lasts over 7 minutes. It doesn’t seem likely that the impact of a team picking up a power weapon first at the start of the match would be very big by the end of the match. Too many other things happen during a match for the initial power weapon acquisition to hold much sway. Another issue is that just picking up the power weapon first doesn’t mean you actually had the power weapon first. On Coliseum, getting Rockets off spawn is a precarious task. It’s an exposed area and it always feels like there’s an endless supply of nades raining down on you. Very often, the team that gets to Rockets first dies immediately after picking them up, without having even shot them once. The other team swoops in and picks the Rockets off the first team’s dead bodies. Even though the other team got the Rockets first, it’s the second team that really gains the advantage of initially possessing Rockets.

Now, there are ways to incorporate this point into the analysis. You could define the first possession of Rockets being the possession when a team actually gets to use them before dying. You could put some kind of time requirement on the possession, saying “only if a team holds Rockets for more than X seconds does it count as a possession.” Or you could be lazy and do what we did and argue that dying with Rockets immediately is part of the hazards of rushing for Rockets. In this way, we don’t manipulate the data: if you get Rockets first, congratulations, you first possessed Rockets, no matter what happens immediately afterwards.

Results

Coliseum is a symmetric map. Therefore, we don’t expect to see a difference in how a match plays out based on which team you’re on. Our expectations were incorrect, and Red Team won much more often than Blue Team in the 1337 matches that we analyzed. We believe this is due to the Halo tradition of the higher seeded team choosing the Red Team color, though we cannot be certain. Table 1 shows this data, along with which team picked up which power weapon first. Notice that Red Team, despite winning most of the time, didn’t pick up power weapons first significantly more often than Blue Team.

Stat Red Team Blue Team Total Matches Won 904 (67.61%) 433 (32.39%) 1337 Pick Up Sniper First 606 (45.33%) 731 (54.67%) 1337 Pick Up Rockets First 713 (53.33%) 624 (46.67%) 1337

Table 1: Matches won and first power weapon picked up, based on Team. Parenthesis indicate percentage of total.

Our actual analysis is not dependent on team color, but instead only on their initial power weapon acquisition status. There are four scenarios possible for any team in any match: pick up Sniper first, pick up Rockets first, pick up both first, or pick up neither first. These are mutually exclusive scenarios, so if you pick up Sniper first, that means the other team got Rockets first. If you pick up Sniper and Rocket first, that counts towards the “pick up both first” scenario. Once we assign each team from each match to one of these four scenarios, we look at how often a team in that scenario wins. These are binary variables. Any team in any match can answer “yes” or “no” to each of the four scenarios, and then “yes” or “no” to “did you win the match?” While there are some advanced statistics you can do with binary variables, we’re only going to do a basic analysis using counting. The data is shown in Table 2.

Scenario Won Match Lost Match Total Pick Up Sniper First 330 (48.46%) 351 (51.54%) 681 Pick Up Rockets First 351 (51.54%) 330 (48.46%) 681 Pick Up Both First 299 (45.58%) 357 (54.42%) 656 Pick up Neither First 357 (54.42%) 299 (45.58%) 656

Table 2: Breakdown of the four initial power weapon scenarios and their win rates.

Notice that if you total the Total column from Table 2, you get 2674, which is double 1337. This is because we’re counting each team in each match as a separate data point, effectively doubling the dataset. Also notice the symmetry: if you pick up the Sniper first and win, that must mean that the other team picked up Rockets first and lost. Remember: if you picked up Sniper and Rockets first, that scenario is called “pick up both first.”

What do these results tell us? Well, first, what we expected: there doesn’t seem to be a huge impact on picking up power weapons first and winning. Sure, if you get Rockets first, you win 51.54% of the time, but that’s hardly a significant value. Of the 681 matches where the teams split the initial power weapons, if the team that got sniper first only won 10 more times, we’d have even results. We’ll go ahead and claim: there is no significant difference in win percentage between picking up Sniper or Rockets first.

The second result is more surprising: the team that gets neither power weapon first wins more often than any other team. Since this is such a basic analysis, it’s hard to say why this is the case. It’s still possible that this is an artifact of a small sample set, but there’s almost a +10% difference in win percentage, implying that it’s a real phenomenon. Our leading hypothesis is that, because we’re only counting the first pick up of the weapon off the pad as the initial possession, other factors may be at play. Perhaps successful teams hang back and get superior positioning on the map to neutralize the power weapon impact, or maybe they’re able to win it back without suffering much at the hands of the other team. Whatever the true cause, we say: there may be an advantage to not being the first team to pick up either of the power weapons.

Part 2: Possession Time

We’ve talked about the weaknesses of using the initial power weapon pick up as a metric for predicting success. Let’s now present an alternative metric that may prove more useful: total time of possession of the power weapon. More specifically, the difference from the average time of possession of the power weapon. Let’s formulate the question like this: “if an average team possesses the Sniper Rifle for X seconds a game, how much do you improve your odds of winning by when you possess the Sniper Rifle for Y seconds more than the average?” This question should account for power weapon possession throughout the entire match, instead of the primitive scope of the question posed in Part 1.

So how do we calculate this? The first step is to tally up the possession time for each weapon for each team in each match. Then calculate the basic stats for these measurements, namely mean and standard deviation. From there, you can do whatever you want.

Results

Let’s get to the good stuff. First, we’ll show the overall stats and the basic counting results. Then we’ll show the more advanced results. But before we get any further, let’s make sure that we understand that possession time isn’t a perfect stat. If a player is holding a weapon, regardless of how much ammo is left, the possession timer is on. We haven’t done any corrections for when a player is running around with an empty weapon with no ammo left. We hope that the ratio of active possession time to total possession time is high, but there’s no way to know as of now. With that being said, let’s get started.

Stat Average (Per Team) Standard Deviation (Per Team) Average For Winning Team Average For Losing Team Sniper Possession Time (seconds) 89.71 61.15 109.59 69.83 Rocket Possession Time (seconds) 57.00 35.23 66.57 47.42

Table 3: Summary of power weapon time of possession stats.

Scenario Won Match Lost Match Total Held Sniper More Than Average 780 (65.60%) 409 (34.40%) 1189 (44.47%) Held Sniper Less Than Average 557 (37.51%) 928 (62.49%) 1485 (55.53%) Held Rockets More Than Average 766 (62.43%) 461 (37.57%) 1227 (44.39%) Held Rockets Less Than Average 571 (39.46%) 876 (60.54%) 1447 (52.35%)

Table 4: Breakdown of power weapon possession scenarios and their outcomes. Note that only the first two and last two rows are mutually exclusive. E.g. a team could have more sniper possession than average, but also more rocket possession than average.

From the basic results shown in Tables 3 and 4, we see what we expected to see: holding either power weapon more than average usually resulted in victory. The split between Rockets and Sniper is close as well, so there’s not too much to be gleaned from these results. Graphically, this is shown below in Figure 1. While this shows a slight advantage to the Sniper, the difference may be insignificant. Our claim is nothing we didn’t know already: holding power weapons longer than the average team is good.

Figure 1: Histograms and best-fit Normal curves for power weapon possession time relative to mean.

The previous stats were calculated by looking at time of possession relative to the average time of possession for each weapon. An alternative approach is to look at the time of possession of a team relative to their opponent in a match. This approach is used below.

Stat Average For Winning Team Standard Deviation Sniper Possession Relative To Opponent (seconds) 39.75 80.04 Rocket Possession Time (seconds) 19.15 49.43

Table 5: Summary of power weapon time of possession relative to opponent stats.

Scenario Won Match Lost Match Total Held Sniper More Than Opponent 948 (70.91%) 389 (29.09%) 1337 Held Rockets More Than Opponent 884 (66.12%) 453 (33.88%) 1337

Table 6: Breakdown of power weapon possession relative to opponent scenarios and their outcomes.

Once again, the results, shown in Tables 5 and 6, are what we’d expect. This time around, however, there’s a bigger gap between the Sniper results and the Rockets results. This is the first real indication that controlling the Sniper Rifle for a longer part of the match is more impactful than doing the same for Rockets. Graphically, the results from above can be seen in Figure 2 below. Our claim: while holding power weapons longer than your opponents is good, holding the Sniper for longer may give you even greater odds of winning than holding Rockets for longer.

Figure 2: Histograms and best-fit Normal curves for power weapon possession time relative to opponent.

We initially planned on doing a typical correlation and linear curve fit analysis to the power weapon possession time data. However, due to the difficulties in dealing with binary variables, the data was modified in a way to make it quasi-continuous (see the Addendum for why binary variables are difficult to work with, and why the correlation coefficient alone is not sufficient). The results of both the unmodified and modified analysis are summarized below in Table 7. The main takeaway here is that, regardless of method used, there is a greater linear correlation between holding the Sniper for longer and winning than there is between holding the Rockets for longer and winning.

Point-Biserial Correlation Coefficient Traditional Correlation Coefficient* Holding Sniper Longer Than Average 0.6501 0.9376 Holding Rockets Longer Than Average 0.5436 0.8914

Table 7: Summary of correlation coefficients. *See addendum on the process to calculate the traditional correlation coefficient.

Conclusion

Our analysis came away with two main conclusions for matches on Coliseum: 1. Getting the power weapons first, either Rockets or Sniper, isn’t hugely impactful on the outcome of the game. 2. It seems that holding the Sniper Rifle for longer in a match, whether that be compared to the average amount of time that a team holds Sniper, or, more impactfully, compared to the opposing team, has a greater effect on the outcome of the match than doing the same with Rockets.

Our first conclusion merely states that it’s not necessary to win the opening battle for power weapons to win the match. Most of you already knew this, but it’s good to see the data back that up.

Our second conclusion means that, if you had to game plan to maximize your team’s use of one power weapon over the other, or if you wanted to minimize the impact of the other team’s use of one power weapon over the other, it probably makes sense to focus on the Sniper Rifle.

Thanks for taking the time to read this through. If you have any questions, comments, or concerns, feel free to post them below.

Addendum

If we were dealing with two non-binary variables, we could then use traditional correlation methods to determine the relationship between the two variables. While we have one non-binary variable, which is the time of possession (it can take on any value from 0 to infinity), we still have win/loss, which is a binary variable. To get the correlation between a non-binary variable and a binary variable, we use something called the point-biserial correlation coefficient.

This calculation gives a value that describes how correlated the two variables are to each other. The value goes from -1 to 1, where -1 means a purely negative linear correlation (increasing one decreases the other linearly), 0 means no linear correlation between the two variables, and +1 means a purely positive linear correlation (increasing one increases the other linearly). It is crucial to note that knowing you have a pure linear correlation doesn’t tell you enough about how much changing one variable changes the other variable. You would then need to look at the linear relationship between the two to judge that. Figure A1 shows why the correlation coefficient alone is not sufficient.

Figure A1: Examples of distributions and the corresponding correlation coefficients. The second row shows that knowing the correlation coefficient is not sufficient to determine the extent of the relationship between two variables (source).

It’s difficult to correlate a binary variable with a non-binary variable, and Figure A2 shows why.

Figure A2: On the x-axis you have a non-binary variable, but on the y-axis you have a binary variable. You can see why it can be problematic to try and determine a relationship between the two.

To get around this problem, we went ahead and put the x-axis variable, Sniper differential time of possession vs. the mean, into a number of bins, as you would do to create a histogram. Then, for each bin, we calculated the win percentage within that bin. So if there were 10 matches in the bin from -100 to -95 seconds, and 7 of them were losses, that bin had a win percentage of 30%. This calculated win percentage became the new y-axis, and the bins themselves became the x-axis. This modified dataset is shown below in Figure A3, and was the source of the traditional correlation coefficient used in Table 7.

Figure A3: Plotting the modified possession time relative to the mean vs. the calculated win percentage.

Though Figure A3 shows a linear fit that implies a better relationship between Rockets and winning, the data set used to generate this was modified and the calculation done may not be reliable. This is nevertheless an interesting result, and perhaps we will look into this further in a future post.