There are countless examples of the Russians using reflexive control. In October 1993, for example, Russian lawmakers took over their own Parliament to advocate for a return to communism. The authorities decided to allow the rebels to occupy a police communications post, giving them access to a secure communications channel then used by police to transmit false conversations between government officials about a plan to storm the occupied Parliament building. After hearing this message, one of the rebel leaders, Parliament Speaker Ruslan Kashbulatov, called on the crowd of supporters to seize a local television station, the first step in a coup. By getting Kashbulatov to make this public request for violence, Russian authorities created justification for storming the Parliament and arresting the dissidents.[10]

Similarly, the East Germans recognized the power of manufactured reality for maintaining internal control. Starting around the 1970s, their Ministry of State Security known as the Stasi expanded the scope of their work from physical abuse of targets—such as torture or executions—to include a certain kind of psychological abuse. The Stasi called this technique Zersetzung, which loosely translates as “decomposition.” It was an organized, scientific effort to collect information about people and then use it in ways that destroyed their sense of self in private and public life. The Stasi broke into the homes of targets to rearrange their furniture, steal items of clothing, or turn off their clocks.[11] They would send compromising photos to loved ones, discredit people in their workplace, remove children from dissident parents or trick them into believing they were mentally ill, something known today as gaslighting. Victims of decomposition struggled to understand why their lives were becoming unrecognizable.

But these Russian and Stasi tactics required careful research and execution to disrupt or manipulate the targets one at a time. The contemporary information environment and modern tools, including artificial intelligence, could slash the transaction costs of such manipulation. The following methods enable a dramatic scaling in the weaponization of information.

The contemporary information environment and modern tools, including artificial intelligence, could slash the transaction costs of such manipulation.

Big Data. By 2025, some predict humans will produce around 463 exabytes each day, enough to fill 212 million DVDs. Their personal data describes individuals with a frightening level of detail. With access to such data from legitimate and illegitimate data brokers, artificial intelligence could combine and match their Amazon purchases, Google searches, tweets, Facebook photos, 401k account balances, credit history, their Netflix viewing habits, and the searches they conduct online searches what they watch on Netflix, etc.

Precision Marketing and Microtargeting. Big data can help identify which individuals are like others—and which similarities matter—based on status updates, drafts of posted videos, facial-recognition data, phone calls, and text messages. Artificial intelligence will make this increasingly easier in the future. If a manipulator knows enough about one person to send him or her messages to provoke or inspire, sending the same messages to similar individuals should produce similar results at scale.

Shallowfakes, Deepfakes, and Social Bots. Shallowfakes are created by manually doctoring images, video, or audio. Deepfakes use artificial intelligence to superimpose images, videos, and recordings onto source files to subtly alter who is doing what. Artificial intelligence-driven “social bots” can carry on conversations as if they were an actual person. Artificial intelligence will enable a dramatic rise in the number of such inauthentic “people” and make it even harder to tell human conversations from artificial ones.

Generative Adversarial Networks. One of the technologies that helps make deepfakes so realistic is the use of a class of machine-learning systems called generative adversarial networks (GANs). These networks have two neural network models, a generator and discriminator. The generator takes training data and learns how to recreate it, while the discriminator tries to distinguish the training data from the recreated data produced by the generator. The two artificial intelligence actors play the game repeatedly, each getting iteratively better at its job. At the moment, generative adversarial networks are being used to build deepfakes for fake porn videos and political satire.[12] But their power to manipulate should worry us for several reasons. First, they can use inputs like big data and precision marketing to scale the manufacturing of content like deepfakes. Second, the iterative competition between generator and discriminator neural networks takes place at machine speed and is accurate. Third, the same logic that underpins generative adversarial networks can be applied to other practices.

Conclusion

Artificial intelligence-driven social bots that are now sending you car or skin care advertisements could start chatting with you—actually, experimenting on you—to test what content will elicit the strongest reactions. Your responses would be fed back into a generative adversarial network-like system where you and countless others play the role of discriminator, all helping the artificial intelligence learn how better to manipulate you in future stimuli. You, or others like you, could slowly be pushed into changing your attitudes, preferences, or behaviors toward other groups or on foreign or domestic policy issues. Whoever is first to develop and employ such systems could easily prey on wide swaths of the public for years to come.

Defending against such massive manipulation will be particularly tricky given the current social media landscape which allows for the easy multiplication of inauthentic individuals, personas, and accounts through the use of bots or other forms of automation. There are several possible ways to develop protective efforts against using artificial intelligence for experimentation on humans. Government regulation is one way. The government could regulate standards of identity authentication by social media companies. This could be done by fining companies that fail to meet these standards, providing tax breaks for companies that meet them, or taxing companies for each user who opens a new social media account. Further, the U.S. Securities and Exchange Commission could develop standards for publicly traded social media companies to report authentic, active users in their filings. In 2016, for example, the Securities and Exchange Commission raised concerns about the ways that Twitter reported Daily and Monthly Active Users on their platforms.[13] The Securities and Exchange Commission could be develop standards of reporting authentic users, which would not only protect investors but also force transparency for who is behind what account on these platforms.