How Russian “Troll factory” tried to effect on Ukraine’s agenda. Analysis of 755 000 tweets

VoxUkraine analyzed over 755 000 tweets that are connected with Russian “Internet Research Agency” and found the evidence of multiple mud-slinging. How is Russian propaganda working?

Author: Oleksandr Nadelnyuk, VoxUkraine Analyst

In the middle of October a "Twitter.Inc" published a dataset with more than 9 millions tweets that “believed to be connected to the Russian Internet Research Agency”. Maybe, such publicity of the tech giant is a result of the investigation on the interference of Russian special services in the 2016 US presidential elections. The published data contain information about 3667 accounts that are associated with Russian “Internet Research Agency” (also known as "troll factory"). We have already written about the influence of Russian twitter-bots on the Brexit referendum: co-founders of VoxUkraine Alexander Talavera and Yuriy Gorodnichenko were the first one who proved that Russians tried to affect Brexit results. We set a goal to investigate whether any of those accounts, which were associated with Russian “Internet Research Agency”, made attempts to influence on the Twitter users thoughts on any Ukrainian issue or event. For this case, we found 755 thousand tweets related to Ukraine using keywords. In this analysis, we have explored the topic of the tweets, defined the most popular events for coverage over the last eight years and found accounts that had been tweeted for almost 33 hours straight. What 9 million tweets got, or how we made the calculations In the dataset, there are 55% of tweets in Russian, 37% — in English, and 8% — in other languages. VoxUkraine decided to analyze how many tweets were directly related to Ukraine. For this case, we, using keywords, filtered out 9 million tweets that are very likely underline the message related to Ukraine. The keywords search helps to detect almost 755 000 tweets. In order to distinguish the messages, we have been searching tweets on hot Ukrainian topics, personalities (e.g. president Petro Poroshenko and ex-president Viktor Yanukovych) and key narratives of Russian propaganda, in addition to different variations of the word "Ukraine". A complete list of marker words in Russian (with translation): «Юго-восток» (South East), «Донбасс» (Donbass), «Боинг» (Boeing), «МВФ» (IMF), «Бандера» (Bandera), «бандеровцы» (banderovtsy), «каратель» (punisher), «ополченец» (pro-Russian fighter), «фашизм» (fascism), «ВСУ» (AFU), «Киев» (Kiev), «революция» (revolution), «Майдан» (Maidan), «Крым» (Crimea), «Донецк» (Donetsk), «Луганск» (Lugansk), «Россия-Украина» (Russia-Ukraine), «Порошенко» (Poroshenko), «Янукович» (Yanukovych), «гражданская война» (civil war), «ДНР» (DPR), «ЛНР» (LPR), «народная республика» (people's republic), «Госдеп» (State Department), «конфликт на Украине» (conflict in Ukraine), «Минобороны Украины» (Ministry of Defence of Ukraine), «MH-17» (MH-17), «ГТС» (Natural gas transmission system), «Тимошенко» (Tymoshenko), «хохлы» (khokhly), «переселенцы» (resettlers). The search period: October 2010 - May 2018. Who published tweets: information about accounts For more than 8 years we have found 774 957 tweets about Ukraine in dataset of "troll factory". They were generated by 1369 accounts (these accounts are now deleted from the social network). Twitter anonymized accounts that have less than 5 000 subscribers, thus we can see only 5% of the user names from the database. The last can be break up into categories: media pages: Vestnyk Moscvy (in English: Moscow Herald, in Russian: “Вестник Москвы”), Vestnyk Peterburga (in English: Petersburg Herald, in Russian: “Вестник Петербурга”) (24 regional “Vestnyks” in total), Federalnoe agentstvo novostey (in English: Federal agency of news, in Russian “Федеральное агентство новостей, Novyny Kyeva (in English: Kyiv News, in Russian: “Новини Києва”)

pages with names: Maksym Dement'ev (Максим Дементьев), Maria Mozhayskaya (Мария Можайская)

thematic pages: Ramzan Kadyrov (Рамзан Кадыров; in the page’s description mentioned as a parody page of Head of Chechen Republic), Myusli Lavrova (in English: Lavrov Thinks, in Russian: “Мюсли Лаврова”), Kholodnaya voyna 2.0 (in English: Cold war 2.0, in Russian: “Холодная война 2.0”). The most followed pages were Vestnyk Moscvy (“Вестник Москвы”) 257 thousand subscribers, Vestnyk Peterburga (“Вестник Петербурга”) — 149 thousand and Maksym Dement'ev (“Максим Дементьев”) — 134 thousand. The total number of followers in 1369 accounts is 3,4 million users. Most of the time, the account owners write (that means we don't know their exact coordinates) that they are located in Moscow (26%) and St. Petersburg (16%), and 17% don't mention their location. When trolls were the most active The first tweet from the sample is dated January 2010, so we have an opportunity to analyze ''trolls" activity before and after the Revolution of Dignity.

Before the start of the Revolution of Dignity in November 2013 and during the next three months, the Twitter trolls haven't shown much interest in Ukraine: the accounts from our sample demonstrated a really low activity. By the spring of 2014, all accounts in total have been publishing an average of 10 tweets per day, and the maximum number of publications for this period was on the Ukrainian parliament elections day on October 28, 2012, — 90 tweets. A mark of 100 tweets per day was reached at the very next day after the "referendum" in Crimea, on March 17, 2014. On this day the accounts published 116 tweets, and on March, 18 — the day of Russia's annexation of the peninsula — 129 tweets. The accounts connected with Russian troll factory have started "working" more productively two days before the presidential elections in Ukraine — 263 tweets were published on May, 23. The next six months the number of tweets was less than 115 per day.

The biggest number of tweets for the eight years was on July 18, 2014 — the next day after MH-17 plane crash. On this day the accounts twitted over 44 thousand messages, and on the next day — over 25 thousand. An eloquent hashtag One of the most effective methods to spread a tweet and to make it reach the Twitter trends accordingly is a hashtag prescription. In our selection, we found over 6200 different hashtags. What are they about? The most popular hashtags in tweets related to Ukraine became #ПровокацияКиева (#KyivProvocation with 22,3 thousand references), #КиевСбилБоинг (#KyivShotDownBoeing with 22,1 thousand) and #КиевСкажиПравду (#KyivTellTheTruth with 21,9 thousand). They appeared exactly on days after MH-17 catastrophe — on July, 18-20. 327 accounts participated in the campaign that promoted these hashtags. Taking into consideration that these are depersonalized accounts, we try to figure out their behavior in the period from July 18 till July 20, 2014.

Hashtag Amount of mentions ПровокацияКиева (KyivProvocation) 22311 КиевСбилБоинг (KyivShotDownBoeing) 22104 КиевСкажиПравду (KyivTellTruth) 21975 Украина (Ukraine) 20640 БитваОлигархов (OligarchsBattle) 10004 новости (news) 8688 Россия (Russia) 8121 СекторБезГаза (SectorWithoutGas) 6882 Крым (Crimea) 6089 КрымПутьНаРодину (CrimeaWayToHome) 3000

Top performer accounts First, what catches the eye is a number of tweets from one account. The most active account from the selection for the period from July 18 till July 19 (no publications on July 20) made 296 tweets and retweets in total. Oddly to say but there are many such "overachievers". If we make a "workaholics rate" on July 18-20, 2014, 277 accounts will get there. They made 196 and more publications in this period of time. Why did we stop on the number 196? Because of the gap of 155 tweets between user №277 and user №278 in the rate. That's why we'll continue analyzing 277 accounts. How one person could make 296 publications for two days? To do this, you should be, at least, a very hardy person. This account has been publishing tweets for 33 hours without large breaks. On average, each post was published every 6 minutes and 42 seconds, and the largest break between publications was only on July 19 from 11:31 till 14:50. As we can see in the visualization, over 200 accounts also had such a "productive" work.

We can assume that several people were managing one account. One of the confirmations of this is the way of publishing the tweets. Usually, users are tweeting directly from the Twitter website or from the mobile app. Such tweets were marked as "Twitter Web Client" and "Twitter for Android/iOS" in the dataset. The tweets can also be published using a special app — a client service. This is a special service for publication and management of one or several accounts. You can use already existent client services or create your own for the work with several accounts. Twitter gives an opportunity for the developers to publish, edit, delete and make other manipulations without visiting the website, only using API. Several services showed a strange activity in July. For example, using a "token_app", Twitter users published their messages only from July 7 till July 19, 2014, and using a "twisofter" — from July 16 till July 19, 2014. With the help of the “twisofter” software during July 18-19 was made the biggest number of publications — 19,3 and 11,2 thousand tweets respectively (or 43% and 40% of the total amount in the corresponding day). "token_app" kept almost the same pace — 18,1 and 10,4 thousand tweets (also 43% and 40% of the total amount in the corresponding day). There are no mentions about these services in the search engines. There is a possibility that these client services were created only for a specific customer or a specific task and weren't promoted to the wide audience as, for example, a new app. Content of the tweets 55% of tweets from almost 755 thousand are unique. It means they are not shared (retweeted) from other accounts. Unfortunately, 98% of tweets were generated by anonymous accounts, so we can't figure out, who shared the content. But we can analyze the pages and the content that was retweeted the most. The most retweeted post belongs to the user with a nickname "_drBre" (now deleted). He's an author of the first and the second (632 and 537 RT, respectively) tweets from our selection that collected a huge number of shares. The third place in the rate belongs to “Pysma na front” (in English: Letter to the frontline, in Russian: "Письма на фронт") (@frontlineletter). In total, the most popular sources of shares among 1369 accounts were Russian media. The first place belongs to RIA Novosti ("РИА Новости") with 23 thousand retweets, in the second place — Federalnoe agentstvo novostey (“Федеральное агентство новостей") with 16 thousand RT and in the third place is "Russia Today" (14 thousand RT). But Russian media didn't collect so many shares per one post. For example, the leader in the total number of retweets, RIA Novosti ("РИА Новости"), had a maximum of 19 retweets per one post. Federalnoe agentstvo novostey (“Федеральное агентство новостей") had 80 RT what is four times more than post from RIA Novosti ("РИА Новости"). However, we are interested not only in the number of retweets, likes or even content. The time of publishing is another confirmation that one person is managing several accounts. 18 out of 19 accounts that shared the RIA Novosti ("РИА Новости") post made it simultaneously — in 16:28 on September 28, 2015. Federalnoe agentstvo novostey (“Федеральное агентство новостей") is in the same situation: the most popular tweet (80 RT) was published by 45 different accounts in 16:34 on July 17, 2014, and by 35 other accounts in 16:43 the same day. Conclusion For a little over 8 years, in the dataset of the "Troll factory", we found 774 957 tweets about Ukraine that were generated by 1369 accounts. Before the annexation of the Crimea, twitter bots showed almost no activity: the maximum number of the publications for the period from 2010 till 2013 was on the Ukraine parliament elections day in 2012 (90 tweets). More actively the accounts from the selection began to tweet at the end of 2014: before the presidential elections (May 23) 263 tweets were published. The next six months the number of tweets was no less than 115 per day.

The real "twitter-storm" was on July 18, 2014 — the next day after the crash of MH-17. On this day the accounts tweeted over 44 000 messages, and on the next — over 25 000. In these days, 297 accounts promoted information that Ukraine is responsible for shooting down the Boeing, using the hashtags #ПровокацияКиев (22,3 thousand references), #КиевСбилБоинг (22,1 thousand) and #КиевСкажиПравду (21,9 thousand).

Over 200 accounts were managed centrally. In addition to the fact that Twitter connects all the accounts from the dataset with Russian “Internet Research Agency”, we found two confirmations that several accounts belong to one owner. There is a strange activity of two client services for working with Twitter since July. For example, the service that is called "twisofter" was used for tweeting only during July 16-19, 2014. The biggest number of publications was made on July 18 and July 19 — 19,3 and 11,2 thousand tweets (or 43% and 40% of the total amount on the corresponding day). Another service that was called "token_app" had the same history. There are no mentions about these services in the search system. In accordance with that, there is a possibility that these services were created only for a specific task.

The second confirmation is the content. 18 out of 19 accounts that shared the most popular for 8 years "РИА Новости" (RIA Novosti) post made it simultaneously — in 16:28 on September 28, 2015. “Федеральное агентство новостей" (Federalnoe agentstvo novostey) is in the same situation. Their most popular tweet (80 RT) was published by 45 different accounts in 16:34 on July 17, 2014, and by 35 other accounts in 16:43 the same day.