With your help, PUBG Labs has just finished its first trial of the Skill Based Rating (SBR) system. A big thank you to all the players who checked out PUBG Labs and took part in this test! We’re pouring through all your feedback and other data from the test so that we can improve the system for the next iteration of SBR coming to Labs.

Some people were curious why we used PUBG Labs to conduct the skill based rating test.

Here are a few of the reasons:

We required large-scale live testing data.

A rating system cannot be meaningfully built in isolation. Similarly to when the Mastery systems were developed, we performed a ‘dark launch’ of Skill Based Rating in order to gather an initial set of data that we used to tune the rating algorithm and create a solid starting point to work off of.

This initial data was not enough however as, among other problems, observing players secretly can’t truly measure how players will perform when they are aware that they are being evaluated. We required real PUBG style games played by real players in real situations over a long-ish period of time to obtain the data required to take the next steps in refining these calculations.



We needed feedback on how the system feels.

Numbers and design philosophies are all well and good, but at the end of the day the system needs to feel good to players. For skill based rating, we wanted to be more open to feedback and work closer with the community than has previously been possible. To that end we included a few different places in Labs for players to directly indicate their feelings on the way the system worked.

Labs gave us the opportunity to share the development of skill based rating early and get a head start on pushing the system in the direction that players want to see.



We wanted to engage the community in the conversation.

We know the community takes skill based ratings very seriously. We felt that it wouldn’t be right to build this in a silo without a solid and up to date idea of ​​what the community wants such a system to look like for PUBG.

Launching skill based rating in Labs jump started the community into a focused discussion around the topic that has proven to be very beneficial to development efforts. So many players shared so many exciting ideas over the course of the test. A few enterprising players even posted up concepts for entire rating systems of their own design! We want all of you to know that we’ve been carefully reading all of your comments and ideas, and will continue to do the best we can to try and deliver a rating system that everyone can be happy with.



Test Cycle Overview

Skill Based Rating went live in PUBG Labs at the end of November and ran for 6 weeks. During this testing period we were able to apply a few adjustments to the algorithm based on what we observed.

https://twitter.com/PUBG_Support/status/1202768163470839808?s=20

Due to the way the infrastructure for Skill Based Rating was built, we were able to make these changes to live servers without needing to bring them down for maintenance, which greatly increased the rate at which we were able to take action to improve the experience.

https://clips.twitch.tv/SpineyHonorablePorcupineOSsloth

After gathering survey results, analyzing the live data, and collecting feedback from various social channels we made a decision to run a second test in February that addresses some of the issues we noticed with the configuration of the algorithm in the first test of Skill Based Rating .

Observations

Next we would like to share some fun and interesting data and feedback we gathered during the skill based rating labs and talk a little about how we plan to move forward into the second test.

18.8% of PC players opted-in to play competitively

22.3% of console players joined competitive ranks of PUBG players (That’s pretty cool!)

Opt-in Percentage by Region