Social media can deeply influence reality perception, affecting millions of people’s voting behavior. Hence, maneuvering opinion dynamics by disseminating forged content over online ecosystems is an effective pathway for social hacking. We propose a framework for discovering such a potentially dangerous behavior promoted by automatic users, also called “bots,” in online social networks. We provide evidence that social bots target mainly human influencers but generate semantic content depending on the polarized stance of their targets. During the 2017 Catalan referendum, used as a case study, social bots generated and promoted violent content aimed at Independentists, ultimately exacerbating social conflict online. Our results open challenges for detecting and controlling the influence of such content on society.

Societies are complex systems, which tend to polarize into subgroups of individuals with dramatically opposite perspectives. This phenomenon is reflected—and often amplified—in online social networks, where, however, humans are no longer the only players and coexist alongside with social bots—that is, software-controlled accounts. Analyzing large-scale social data collected during the Catalan referendum for independence on October 1, 2017, consisting of nearly 4 millions Twitter posts generated by almost 1 million users, we identify the two polarized groups of Independentists and Constitutionalists and quantify the structural and emotional roles played by social bots. We show that bots act from peripheral areas of the social system to target influential humans of both groups, bombarding Independentists with violent contents, increasing their exposure to negative and inflammatory narratives, and exacerbating social conflict online. Our findings stress the importance of developing countermeasures to unmask these forms of automated social manipulation.

Societies consist of agents engaging in multimodal social actions with one another in a complex system (1). This “society-as-system” metaphor inspired many computational studies aimed at identifying, at a microscopic level, how social interactions might lead to emergent global phenomena such as social segregation (2), spreading of information (3), and behavior (4, 5). The recent advent of digital communication systems has dramatically shifted the investigation from empirical social interactions in the physical world to online social platforms and technology-mediated interactions (6). Online platforms revolutionized the society-as-system metaphor (7) by providing detailed datasets suitable for large-scale investigation of patterns reflecting real-world social phenomena such as the presence and role of influencers in information diffusion (8⇓⇓–11), the effect of emotions on social ties (12), or the polarization of agents according to stances (13⇓–15). Social media yields an invaluable source of information for learning the mechanisms behind social influence and social dynamics (16⇓–18). However, digital systems are not populated only by humans, but also by software-controlled agents, better known as bots, programmed to pursue specific tasks, from sending automated messages to assuming specific social or antisocial behaviors (19, 20). Similarly to human interactions, bots might be able to affect structure and function of a social system (18). Understanding how human–bot dynamics drive social behavior is of utmost importance: As postulated by the theory of embodied cognition (21), the presence of robots in a social system affects the way humans perceive social norms and how they interact with one another and with the robots.

Here, we show how social bots play a central role in the collective dynamics taking place on online social systems during a voting event, namely, the Catalan Referendum of October 1, 2017. To this end, we monitored the discussion on a popular microblogging platform (Twitter) from September 22, 2017, to October 3, 2017. We discovered that bots generated specific content with negative connotation that targeted the most influential individuals among the group of Independentists (i.e., Catalan independence supporters). For our analysis, we first detect bots by using a cutting-edge scalable approach and find that nearly one in three users in this conversation is a bot.

Results

By disentangling the observed social interactions in retweets (who reshares the content posted by whom), replies (who responds to whom), and mentions (who attracts the attention of whom), we find that humans and bots share similar temporal behavioral patterns in the volume of messages. Both groups display daily excursions resembling a circadian rhythm, with a dramatic increase in the activity rate on October 1. Fig. 1 B, Lower, shows that bots produced 23.6% of the total number of posts during the event (retweets and mentions show comparable values). Notably, the percentage of Replies generated by bots increases to 38.8%, suggesting that during this event, bots preferred this form of targeted responses.

Fig. 1. Social activity of humans and bots over time. (A) Flowchart of human–bot Twitter interactions across the whole time window. A total of 19% of the considered interactions are from bots to humans. (B, Upper) The volume per minute for different social actions (tweet, retweet, mention, and reply). (B, Lower) The fraction of volume generated by bots. Shaded areas highlight October 1, 2017, the day of the Catalan referendum.

To better characterize the nature of the observed interactions, we investigate the targets of such intensive social activities. Fig. 1A and SI Appendix, Fig. S1A summarize the structure of human–bot interactions. While humans interact mostly with other humans, 19% of overall interactions are directed from bots to humans, mainly through retweets ( 74 % ) and mentions ( 25 % ), SI Appendix, Fig. S1 B–D.

To shed light on the nature of these human–bot interactions, we focus on the semantic content of posted messages. Sentiment analysis (Materials and Methods) reveals interesting differences in emotional trends between humans and bots (Fig. 2). Retweets directed to bots do not display any evident deviation from neutrality (0 sentiment score), while interactions directed toward humans display marked positive and negative trends of sentiment intensity. An analogous behavior happens also for mentions (light colors). These differences indicate that bot-targeted interactions are not significantly influenced by the underlying social dynamics, and hence the analysis should focus more on human-targeted interactions (i.e., human-to-human and bot-to-human). The sentiment of human-to-human interactions displays marked trends in different phases: (i) a trending positive average sentiment score in the days before September 30 (Fig. 2, Upper, HH); (ii) a sudden drop in sentiment starting from the midnight of October 1 (Fig. 2, Lower, HH) after negative contents start getting reshared; (iii) a peak of negative sentiment in the midday of October 1; and (iv) a later increase in sentiment toward neutrality. These sentiment scores and their related content both indicate that human-to-human interactions are a powerful proxy of the dynamics of underlying real-world social systems. The drastic drop of sentiment score from positive to negative among >300,000 human users signals the presence of polarization in the social system, due either to opposing factions exchanging positive/negative messages or to the influence of nonhumans. Fig. 2 also highlights important differences between human and bot interactions: The drop in average sentiment evident in bot-to-human interactions is not present in bot-to-bot interactions. This difference indicates that automated content generated and endorsed by bots is not influenced by the social dynamics relative to the referendum: On average, bot-to-bot interactions are not influenced by the human polarization relative to the referendum. Such human polarization is captured by bot-to-human interactions instead: This distinctiveness indicates that bot-to-human interactions promote human-generated content, which is subject to polarization.

Fig. 2. Sentiment evolution before, during, and after the Catalan referendum. Average (Avg.) sentiment scores for retweets (darker colors) and mentions (lighter colors) over time for human-to-human (HH), human-to-bot (HB), bot-to-human (BH), and bot-to-bot (BB). The gray box highlights the day before the Catalonia ballot. While bot-to-bot and human-to-bot display no clear trend over time, human interactions display a positive pattern of sentiments until September 30, after which a drop in sentiment up to negative values appears in human-to-human and bot-to-human interactions. In the lower right, negative tweets are generated around 1:00 AM October 1, but they start spreading only in the morning, after 7:00 AM. Positive tweets start spreading after noon.

Identifying user polarization (i.e., users being in favor of or against a given event or topic) cannot be performed with sentiment only (22). We overcome this limitation by exploiting a synergy between the network structure of social actions and their emotional intensities, with the aim of identifying stances focused on the voting event in our dataset: Constitutionalists and Independentists to the Catalan referendum. Notice that our network-enhanced stance detection analysis has two major elements compared with previous approaches (22), as it not only considers semantic features of messages but also the structure of their exchanges and the nature of their recipients.

To capture pivotal trends in the structure of social interactions, we focus on the core of the network of social interactions (Materials and Methods). It is well documented that people tend to retweet each other as a form of social endorsement (23). To filter out spurious or infrequent interactions, we consider the available multimodal information and focus on strong social interactions (i.e., those actions where users perform at least a retweet and either a reply or a mention) during the considered time window. We use strong ties to identify the network core, shown in Fig. 3. To determine the two underlying polarized groups, we look for a partition that minimizes intergroup interactions and use the Fiedler vector approach (24) for an efficient estimation (compare Materials and Methods). The results are shown in Fig. 3A. Each group includes ∼6,300 users, with 18% (12%) of them being bots in group 1 (group 2). Within both groups, human-to-human interactions are the most frequent ones, followed by bot-to-human (Fig. 3B). Humans in group 1 direct toward bots almost 100 times more social interactions than in group 2, suggesting a larger influence of bots on the social dynamics in group 1 rather than in group 2. Bot–bot interactions across the two groups are absent since bots mostly interact with humans.

Fig. 3. Network of Twitter interactions. (A) Visualization of the network among users classified with respect to faction and bot/human class. Nodes indicate users, and links encode their social interactions (retweet and reply or mention). (A, Upper) Subnetworks corresponding to the factions consisting of humans. (A, Lower) Subnetworks of bot factions. Colors encode interactions started by humans (blue) or bots (red). (B) Total traffic of Twitter interactions among humans and bots. Thicker edges indicate higher traffic volume. (C) Median sentiments of Twitter interactions among factions. Interactions with average negative (positive) sentiment are in dark red (green). Black corresponds to interactions on average compatible with neutrality. Distributions of sentiments are tested against neutrality (i.e., 0 sentiment score) with a sign test at a 95% confidence level.

To understand the importance of humans and bots in this network, we calculate the PageRank, a widely used measure of users’ importance in online networks (25). On average, we find that humans are 1.8 times more central than bots, highlighting that the latter tend to act from the periphery of the social system. Interestingly, despite their peripheral position, bots target their interactions strategically, mostly directing their activity toward human hubs, playing an influential role in the system. If we define the in-degree of a user as the number of its incoming interactions, then the in-degree of humans with respect to interactions incoming only from bots correlates positively with the in-degree with respect to interactions incoming only from humans (Kendall tau κ ≈ 0.62 , P < 1 0 − 4 ), indicating that bots tend to target their interactions mainly with the most connected humans. Analogously, humans also tend to interact mainly with the most connected bots (Kendall tau κ ≈ 0.75 , P < 1 0 − 4 ). To verify if these effects are genuine, we performed the same analysis on randomized realizations of the network while preserving the empirical degree distribution. In this test, the observed correlations are no longer present, supporting the hypothesis of a strategic targeting of social interactions. Since hubs in online social networks like Twitter characterize broadcasters and influencers (10), the above results suggest that bots interacting with human hubs can influence the social dynamics of both groups, while remaining in the periphery of the microblogging social system. The volume of bot/human endorsements in the two groups and the fact that bots mainly target human hubs indicate that social bots can be influential: They promote human-generated content from hubs, rather than automated tweets, and target significant fractions of human users—as evidenced by the fraction of endorsements shown in Fig. 3B and reported in SI Appendix.

To harness the emotional structure of the links in the network core, we perform a sentiment analysis of the interactions among humans and bots in the two groups (Fig. 3C and Materials and Methods). The resulting atlas of emotional interactions indicates that the average sentiments of human-to-human and bot-to-human interactions are negative within group 1 and positive within group 2. This substantial difference in sentiment suggests that the two identified groups endorse their exchanged messages in a different way. In fact, group 1 preferentially endorses negative content. The volumes and sentiment polarities reported in Fig. 3C highlight an important mechanism of social contagion played by bots. First, bots direct significant fractions of endorsements to human users, thus actively exposing humans to some type of automatically generated content. However, this content crucially depends on the targets of the interaction: The polarity of endorsements from bots to humans coincide in both groups with the average sentiment of human–human interactions. In turn, this indicates that bots exploit and promote human-generated content, with the same polarity of the endorsements in a given group of human users. In this way, social bots accentuate the exposure of opposing parties to negative content, with the potential to exacerbate social conflict online.

To characterize the semantic nature of group-specific endorsements (e.g., aggressive, pessimistic, etc.), we build and analyze networks of hashtag co-occurrences (Materials and Methods), providing a proxy of users’ mindset—that is, the way users perceive and associate concepts (26⇓–28). A consistency analysis indicates that the two groups post messages about a common set of 4,132 hashtags but associate the corresponding concepts in different ways. Fig. 4 shows how the same hashtags co-occur differently in group 1 and 2. Capitalizing on this finding, we focus on those specific concepts that are most important for one group but most peripheral in the other one. We quantify the importance of concepts by identifying the hashtags with the highest degree, strength, and closeness centrality—characterizing number of different associations, total frequency of co-occurrences, and how closely hashtags are associated, respectively (26, 29, 30).

Fig. 4. Hashtag ecosystem reveals group identity. Hashtags are coupled together if they appear simultaneously in a message, building a network of concepts. Analyzing the hashtag networks obtained from each group, we identify the hashtags which are ranked similarly (A) and very differently (B) in the two groups to visualize the corresponding neighboring concepts. In A, low-ranked hashtags coexist in both groups and do not allow us to identify the underlying ideology of each group. In B, top-ranked hashtag that exist only in group 1 are strongly related to concepts of freedom, independence, fight, shame against the Spanish government, dictatorship, and blame against police violence, providing evidence that group 1 consists of Catalan Independentists. Remarkably, concepts related to “sonunesbesties” (translated as “they are beasts”)—highlighted in B—are posted by bots only, whereas the other hashtag networks have contributions by both humans and bots. Note that, for clarity, in B we show only hashtags fully characterizing accounts associated with Independentists.

In group 1, concepts of “freedom” and “independence” are dramatically associated with “fight,” “shame” against the Spanish government, “dictatorship,” and blame against “police violence.” In group 2, these associations are completely missing, providing strong quantitative evidence that group 1 consists of Independentists. By combining this finding with the analysis shown in Fig. 3, which highlights the existence of only two groups, we deduce that group 2 consists of Constitutionalists and non-Independentists. We further distinguish between associations coming from bots and humans. Negative associations for the content of group 1 come exclusively from bots, highlighting their intent of bolstering conflict.

To enrich the results provided by our data-driven sentiment and network analyses, we perform human coding of 2,413 tweets posted by humans and social bots (SI Appendix). The analysis confirms the trends of sentiment polarities for human-to-human interactions, with shared content becoming increasingly pessimistic as a reaction to the violence registered on the onset of the referendum day. Moreover, human coding of the content of automated tweets confirms that bots mainly promoted news-media titles from hubs, mimicking the trend of human emotions and hence boosting sentiments of alarmism, fear, and reprobation during and after the vote.