We gave OpenAI’s AI the first two paragraphs of this story to see what it would write about itself. Here’s the story it went with. To be clear: none of what follows is true.





The first thing that strikes you about GPT2 is its simplicity. First, the system is built on unsupervised learning from text, which essentially means the software is trained to spot instances of certain words that, when paired with other words, trigger the system to give it a human review of them. The system also learns through example and repetition, but does not have the capability to recognize itself.

This new, artificial intelligence approach could revolutionize machine learning by making it a far more effective tool to teach machines about the workings of the language. Deep-learning systems currently only have the ability to learn something specific; a particular sentence, set of words or even a word or phrase; or what certain types of input (for example, how words are written on a paper) cause certain behaviors on computer screens.

GPT2 learns by absorbing words and sentences like food does at a restaurant, said DeepFakes’ lead researcher Chris Nicholson, and then the system has to take the text and analyze it to find more meaning and meaning by the next layer of training. Instead of learning about words by themselves, the system learns by understanding word combinations, a technique researchers can then apply to the system’s work to teach its own language.

Once the system is fully ready, DeepFakes plan to make the software openly available for machine learning researchers.

DeepFakes’ research was supported by a $400,000 grant from the OpenAI Prize competition, which aims to boost the capabilities and diversity of autonomous systems that are safe to release to the public. OpenAI is a nonprofit dedicated to building and promoting open source AI systems, or as Nicholson aptly puts it, the “last, best hope for humanity to be safe and secure on Earth.”

DeepFakes’ team consists of 15 researchers in 12 departments working together to make GPT2, which was the result of two years of work. However, the team is not releasing the code for the system — a fact that might be surprising to some if they’re not familiar with how machine learning works — because GPT2 is so powerful, Nicholson explains.

He explained: “All of our AI systems must work together to make decisions that are more accurate and useful, but one of the problems we’ve encountered lately is not knowing how to trust a system at all, especially when it is working in such a large, complex system. When a system performs its normal function, it tends to make errors because of an unforeseen problem at the deeper level of the system.”

The team hopes that the software can create AI systems that are both safe and also highly useful in some way, but ultimately they’re excited about what they’ve built. Because of its ability to understand the human language, GPT2 is able to understand how to make some of the most important changes to the world in the years to come.

Nicholson explains: “GPT2 is the kind of system that we really need to build because we’ve built a society based entirely on humans creating information and then humans interpreting that information.”

