The Science and Math

Consider a simple spin-bot cheat used by a player in the game. The aim of the player is spinning haphazardly and for a split second aims at the enemy and shoots them when they are in sight. Considering overly simplified statistics (just for the sake of understanding the problem), we can check the crosshair movement and position of the player to determine if they are using this spin-bot or not. But this movement can be mimicked by a player who is not cheating by using extremely high sensitivity or if the player’s mouse movement bugs out. Now we need to consider other things like movement, difference in time between when the player snaps him aim for a split second and gets the kill. One can build a statistical model by considering these features (and many more) to detect a genuine player messing around with sensitivity and an actual spin-bot. But these models are easy to reverse engineer. As mentioned in the comment by Valve, “any hard-coded detection of spin-botting leads to an arms race with cheat developers — if they can find the edges of the heuristic you’re using to detect the cheat, the problem comes back.” Once they know how the model is developed and what features were used they can try and circumvent those measures. The model is not going to change hence the reverse engineering is successful in evading that model.

Enter Machine Learning, if the model keeps learning about these counter measurements then the anti-cheat gets the upper hand in this arms race with the cheat developers. Machine Learning models use various algorithms that try to mimic how human brain learns things. There are several algorithms that are helpful to classify data into two categories. The two categories we are talking about here are cheating and not cheating.

A probable Machine Learning flow for CS:GO

The above flow diagram tries to explain how the approach is being currently tried out by Valve. Please note that this might not be accurate and is extremely generic. This flow of Machine Learning algorithms are split into three different phases here: 1. Training, 2. Testing and 3. Validation.

In the training phase CS:GO demos are parsed for various features and fed to the algorithm. Example features can be: hit accuracy, player movement, crosshair movement, fire on aim ratio, fire on visible ratio (when the enemy is visible), time to hit, etc. Redittor /u/PM_ME_SOME_STORIES attempted to implement a neural network algorithm and they have also given a couple example papers to study which kind of features can be useful to train the model. Each of these features have weights that describe how important of a factor they are in determining if a player is cheating or not. For example, hitting headshots and time to hit are important factors so they have a chance of having a higher weight factor. All these weights are calculated during the training phase. Each demo instance that is ran through the model changes the weights according to if the demo contains cheating or not. Valve has access to thousands of demos of people cheating and not, which is an extremely rich dataset to start with and this helps to train an efficient model.

Once the model is built by using sample demos, it is time to test it. Another chunk of sample demos is fed to the model. This time the feature if the demo contains cheating or not is hidden from the model and the model has to determine if there was any cheating or not. The model uses the features and weights that it has learned in the previous training stage and classifies these samples into the two categories. If the demo is determined to contain cheating then it is sent to review in the Overwatch system where the community investigators can re-affirm the inference made by the model.

The results from the model and Overwatch are then assessed during the validation stage. Here checks are made to improve the model, false positives are fed back to the training model and the model keeps learning and improving in accuracy. A continuous loop operation of these three stages helps to keep up to date with the most recent cheats and even the counter measures taken by cheat developers.