Source: http://forum.worldoftanks.eu/index.php?/topic/468409-a-1000-battle-study-on-xvm-win-chance-accuracy/

A 1000 Battle Study on XVM Win Chance Accuracy.

Author: Spitfeuer117

Published with author’s permission.

If you have XVM with statistics enabled in it, it will give a ”Chance to win” percentage for the battle. I went on a quest to find out whether you would actually win that battle that many times out of a hundred.

The only way to be sure would be to play the exact same battle thousands of times under the same conditions. Instead, I started to evaluate my win percentages under the battles with the same XVM predictions.

Out of 1000 battles, there were 598 wins, 390 losses and 12 draws. The average XVM prediction was 55%, which seems low compared to the 60% win rate.

By sorting the number of wins, losses and draws into the XVM prediction categories, the following histogram was produced:

By plotting the win rate in each category as function of the XVM prediction, we get the following figure. Only the categories with more than 20 battles (44%-66%) were taken into account. Also, draws were ignored as XVM does not take them into account in win chance calculation, i.e. two equal teams have both 50% predictions.

From the graph we can see that at 50% XVM gives a correct result, indicating that it calculated my skill (or my impact in the games) well. However, the higher win chance predictions are increasingly too low and the lower are increasingly too high.

XVM calculates the win chance by first calculating a skill value for each player by using a large number of statistical variables. It then calculates the skill of each team, Ka and Ke, recorded in xvm.log, by adding the player values together. The win chance is then calculated using the following formula:

P(win) = ((Ka / (Ka + Ke) – 0.5) * 1.5 + 0.5) * 100%

If P(win) > 95%, P(win) = 95%. [1]

Basically it scales the skill percentage with an arbitrary factor of 1.5. According to the figure above, the scaling factor should in fact be 2.3 times higher, 3.5, to produce more accurate results. Win chance as function of the skill percentage with the different scaling factors would look like this:

The correct way of determining the probability of a victory is very different however. If we assume that player’s effectiveness in a battle is the sum of infinitely many variables, it and therefore also the team’s effectiveness is normally distributed with an average of the skill value calculated by XVM for example. If we then assume that the winning team is the one with a higher effectiveness in the battle, the win probability is

P(win) = Φ((Ka – Ke) / s) * 100%

where Φ is the cumulative distribution function of the standard normal distribution and s the standard deviation in the teams’ skill difference. The standard deviation can be assumed constant and determined experimentally. A value of s = 30 seems to provide correct results. Figure below shows the shape of the function.

Note: Different horizontal axis. The use of skill difference instead of ratio brings low tier low skill battles closer to 50%, which makes sense. For example the figure below shows win chance with the normal distribution formula and XVM formulas with a scaling of 1.5 and 3.5 as function of enemy team skill, if allied team skill is 1.5 times higher.

I tested the normal distribution formula with the same method with 324 battles, and the following results were produced:

Although the number of battles was small, the normal distribution method provided roughly correct results. My win rate during the 324 battles was 61% instead of 60%, which is probably why the results are slightly higher than expected.

Conclusion

XVM is inaccurate. You win for example 27% of 40%ers, 50% of 50%ers and 73% of 60%ers.

[1] http://www.modxvm.com/en/faq/how-is-the-teams-chance-to-win-calculated/