In this work, we analyze what effect streaming gameplay on Twitch has on players’ in-game behavior and performance. We hypothesized that streaming can act as a form of implicit incentive to boost players’ performance and engagement. To test this hypothesis, we continuously collected data about all Twitch streams related to a popular Multiplayer Online Battle Arena (MOBA) game, League of Legends (LoL), and data of all LoL matches played during the same time frame, and cross-mapped the two data sets. We found that, counterintuitively, streaming significantly deteriorates players’ in-game performance: This may be due to the burden of carrying out two cognitively intensive activities at the same time, namely, playing the game and producing its commentary for streaming purposes. On the other hand, streaming increases engagement keeping players in significantly longer game sessions. We investigate these two effects further, to characterize how they vary upon individual characteristics.

Introduction Game streaming platforms like Twitch play a pivotal role in the growing popularity of esports, accounting for a huge cohort of players that daily broadcast their gameplays live, attract viewers, and gather donations. They not only allow players to share their content online but also constitute a possible source of engagement in the game for both players and their audience. On the one hand, many studies have focused on human engagement in streaming platforms, as well as trying to understand the motivation that users have to stream (Hilvert-Bruce, Neill, Sjöblom, & Hamari, 2018; Hu, Zhang, & Wang, 2017; Sjöblom & Hamari, 2017), and their behavioral patterns on such platforms (Hamilton, Garretson, & Kerne, 2014; Lessel, Mauderer, Wolff, & Krüger, 2017; Zhu, Yang, & Dai, 2017). On the other hand, engagement and motivation has been analyzed from the game perspective, by studying what are the characteristics of the game that drive users to be more engaged, such as the game ranking systems (Kou, Gui, & Kow, 2016), team composition (Kou & Gui, 2014), and that retain them over an extended period of time in online games (Park, Cha, Kwak, & Chen, 2017). However, whenever a game is streamed, the user is actually engaging on two different platforms: the streaming platform and the game platform. Thus, to better understand what are the factors that affect players in both their engagement and performance when streaming, we need to consider both the streaming and the game platform and identify any change in their behavior that would lead to better performance and engagement. In the present work, we aim at shedding light on the effects that streaming has on players’ performance and engagement by taking into account their behavior in different conditions: streaming and nonstreaming. To this aim, we collect data about players in a popular Multiplayer Online Battle Arena (MOBA) game: League of Legends (LoL). Since its release in 2009, LoL has not only attracted the attention of millions of users that regularly play on the platform but has also become one of the most streamed online games on YouTube and Twitch.tv. Due to its popularity and huge cohort of streamers, we focus on the study of players’ performance and engagement in LoL and how they are affected by streaming on Twitch. Moreover, the accessibility to both streamed data and in-game data allows us to compare how streamers’ behaviors change when streaming and nonstreaming as well as to study the differences between streamers and nonstreamers. We are particularly interested in identifying the effects of streaming on performance and engagement at different levels. First, we study the effect of streaming in the long term, by comparing the level of engagement in LoL of streamers and nonstreamers over the entire observation period. Second, we investigate if streaming leads to longer engagement in the game in the short term, namely, during an individual match and over the course of a session, that is, a sequence of matches played consecutively without an extended break. We use an analogous analysis to study the performance dynamics over time and corroborate previous results which show how performance deteriorates over game sessions. Here, we also try to disentangle different aspects that can influence players’ performance, for example, popularity and skill level. Finally, we use a mixed effect model to test our hypothesis about streaming impact on performance. The article is organized as follows. In second section, we introduce the platforms studied and explain our data collection process. In third section, we summarize the methods used throughout the article to analyze the impact of streaming on players’ performance and engagement. In fourth section, we outline the results obtained in our study and report the work relevant to our findings in fifth section. Finally, we report our main findings and related conclusions in sixth section.

Method In the following, we describe the methods used in the present study to understand both a player’s engagement and performance and how these two aspects are affected by streaming. On the one hand, to study the effect of streaming on players’ engagement in the game, we compare how much time players spend in their matches (when streaming or not), by both looking at the average duration of each match and how many matches in a session they play. On the other hand, we compare performance over the course of sessions of different lengths in three scenarios: Twitch user sessions (streamed and nonstreamed) and non-Twitch user sessions. Finally, we investigate players’ characteristics and their relation to in-game performance. Average Match Duration Difference The first metric we use to investigate a player’s engagement is the average match duration of both streamed games and nonstreamed games. To compare the two cases, we compute the difference Δi for each user i as: Δ i = ∑ t = 0 M x t i , l = streamed M ∑ t = 0 N x t i , l = nonstreamed N , 1 where N is the number of streamed matches and M is the number of nonstreamed matches of player i. Δi is then the average game duration difference of user i, which is positive if he or she spent more time in streaming matches, and negative otherwise. Survival Rate The second metric used to understand users’ engagement is the survival rate, which we define as the probability that a user will play another match after the last one. This analysis allows to identify different levels of engagement between players and their relation to streaming. In particular, we can distinguish between a long-term and shot-term survival rate. The former refers to the probability of having, in the life span of our data set, a certain survival time, which is the temporal distance (measured in days) between the last and first match played in the player’s whole records. Thus, if N days separate the first and the last recorded match, then a player’s survival time will be N. The latter is defined as the probability that a player will start a new match after another one in the same session. Analogously to the long-term metric, here, we consider the distance between the last and first match of a session. However, the distance is measured by the match index in the session and thus by the session length, that is, the number of matches in a session. The computation of the survival rate in both short term and long term helps us understanding the effect of streaming on a player’s engagement in the game. By studying the long-term effect, we can indeed measure how long streamers keep playing the game in comparison with nonstreamers, while the short-term effect highlights differences between streamers and nonstreamers (streamed sessions vs. sessions of players not using Twitch) and effects of streaming on individuals (streamed vs. nonstreamed sessions of Twitch users). Performance Indicator: KDA To investigate the consequences of streaming on users’ performance, we study how a user’s performance changes over time, and in particular over the course of sessions of different lengths. To this aim, we use a proxy for in-game performance that is popular between MOBA game players: the KDA ratio. The KDA ratio can be computed as follows: KDA = # of kills + # of assists max ( 1 , # of deaths ) . 2 In a nutshell, the KDA is a ratio between qualitatively positive actions that a player performs during the game (killing enemies and assisting teammates) and negative actions that are harmful to the player and his teammates (champion’s deaths). Thus, if a player kills a lot of enemies (or assists teammates in doing so) but he or she also dies very frequently in such exchanges, the final KDA score will be lower than a player who actually manages to kill or assist while staying alive during these fights. Performance Over the Course of a Session Once the KDA of each match in a player’s history is computed, we can study how it changes over the course of a session. To this aim, we report the KDA transition from the first to the last match of a session, for sessions of different lengths and user categories. It has been indeed shown in previous works (Ferrara, Alipourfard, Burghardt, Gopal, & Lerman, 2017; Kooti et al., 2016; Singer et al., 2016) that users’ performance tends to deteriorate over time due to mental fatigue and the higher effort in keeping focus after a certain period of time. In the following, we compute the average KDA over sessions of the same length in our data and plot them to observe this phenomenon. We are particularly interested in understanding whether performance deterioration differs in relation with a player’s category (streamers vs. nonstreamers) and their characteristics: popularity and skill. Here, the popularity of a player is provided by the number of followers on Twitch, while the skill is computed as the player’s average KDA. On the basis of both popularity and skill level, we can further distinguish between players that have high/low popularity and high/low skill level. Mixed Effect Models Finally, we aim at identifying the relation between a player’s performance and his or her characteristics. To this aim, we use a mixed effect model. Our hypothesis is that streaming and game session length both affect player performance. Streaming might indeed increase players’ engagement in the game, as streamers have to demonstrate their abilities in front of an audience. Furthermore, players’ performance may deteriorate over time due increased mental fatigue effects. The mixed effect model allows us to assume these two factors to be heterogeneous among users, by incorporating both fixed and random effects. Given a condition l with l 2L where, L = {session length, streaming}, a user i, and his or her vector of observations y i we can compute: E l ( y i ) = β l X l , 3 where β is the fixed effect for condition l. Given the two conditions, we can then write our model for each user i as follows: y i = ∑ l ϵ L E 1 + γ 0 i + β 0 + ϵ i , 4 where β 0 and γ i0 are, respectively, the fixed and the random effect intercepts, and ϵ i is an unknown vector of random errors. Note that the parameter γ varies depending on the user, while β is fixed for each user in the data set. There are different possible configurations for the mixed effect model, in which we can decide to add a random effect to either one variable X l or to all of them. Given the condition l in which a random effect is applied together with the fixed effect, its expectation will be: E l ( y i ) = ( β l + γ l ) X l . 5 We tested all the four possible combination of the model. However, as explained in fourth section, we did not find any relevant difference in the results of the different models. Thus, we chose to use the simplest model, where we define the expectation as in Equation 3 under both conditions l. This not only is the simplest model but also does not require us to further manipulate our data as needed in the other models.

Conclusions In the present work, we studied the effects that streaming has on both a player’s performance and engagement in a game. To this aim, we collected data of players streaming LoL, one of the most streamed MOBA games on Twitch, as well as their match information and history through the official Riot Games API. This data set allowed us to study not only the changes in streamers behaviors, by comparing their performance and engagement in streamed and nonstreamed matches, but also the difference between streamers and nonstreamers (we collected additional data of LoL players that do not stream on Twitch). First, we analyzed engagement differences in these categories of players. We found that streaming a match has a positive effect on players’ engagement: Streaming a match leads to play longer matches, and streamers tend to have higher engagement both in the short term and in the long term. We indeed identified that streamers engage in longer sessions than in the case in which they do not stream, thus displaying higher survival rates. This is also true when comparing streamers and nonstreamers. Moreover, we observed higher survival rates in the long term: Streamers tend to engage in the game for long periods of time and have a longer player history than nonstreamers. Second, we analyzed the effects of streaming on players’ performance. We noticed that there is no difference in the win rate of streamed and nonstreamed matches. However, the variable “win” can be biased by other aspects of the game such as composition of teams, teammates abilities, in-game matchmaking design, and so on. Therefore, we used the KDA ratio as a proxy for a player’s performance. The KDA ratio of a player indeed reflects the actions that the player performs in each match and thus provides an estimator of the quality of playing in the match. We studied performance over the course of a session, for each of the three session types: streamed session, nonstreamed session, and sessions of players that do not stream their matches. The results, in line with prior literature, show that in general players are subject to performance depletion when playing a sequence of matches without an extended break. Performance over the course of a session does decay; however, we observed that streamers are more affected by this mechanism when streaming than in the case in which they are not streaming. This result suggests that streaming might be a source of distraction for players, taking cognitive bandwidth away from gameplay, as players often comment and try to engage with their audience while streaming. To detect possible factors connected to performance decay, we further analyzed sessions of players segmented by popularity, that is, number of followers, and skill, that is, KDA levels. The results of our analysis led to the conclusion that performance decay is not affected by the number of followers one player has. However, if a player has higher skills, his or her performance does not reflect the typical performance depletion pattern. This mechanism is indeed mitigated by the fact that higher skill players most likely can keep focus on the game for longer time while producing the streaming commentary at the same time. Finally, we studied what players’ characteristics have the strongest effect on performance. In particular, on the basis of our previous results, we assumed that both the match position in a session and streaming can negatively affect players’ performance. To test this hypothesis, we used a mixed effect model that allows us to incorporate heterogeneous effects of these two gaming aspects among players. We confirmed that streaming has a negative influence on a player performance even if it boosts engagement in the game. In conclusion, through the combined study of Twitch and LoL data, we found that streaming has a major impact on a user engagement in the game, by making users both playing longer and more regularly over time. We have also shown how performance deteriorates over sessions, thus corroborating the extant literature. However, we found that performance decay is mitigated in high-skill players, and that it is not affected by streamer’s popularity.

Authors’ Note

This project does not necessarily reflect the position/policy of the government; no official endorsement should be inferred. Approved for public release; unlimited distribution.

Acknowledgments The authors are grateful to DARPA for support (grant #D16AP00115).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article. ORCID iD

Akira Matsui https://orcid.org/0000-0003-3953-378X

Notes 1.

In the top 3 of all platforms in June 2018.