Facebook CTO Mike Schroepfer on how the social network is building machines that will help us plan our daily lives.

Most days you learn something new: that your train to work is habitually five minutes late or that you can no longer disguise your fast-retreating hairline.

In response, you resign yourself to winding back your alarm and investing in a buzzcut. Unfortunately machines are typically less pragmatic, incapable of learning as they go and reassessing their approach to the world in this way.

Once their instructions have been issued computers will follow them to the letter, even when circumstances dictate they should do otherwise.

Even systems deemed capable of learning generally absorb information in a very different manner to people - learning everything up front during training sessions in which they ingest huge amounts of human-labelled data or pick out recurring patterns from datasets.

Image: Facebook

"Post training it doesn't learn anything. It's kind of fixed in its capability and what it does," said Facebook CTO Mike Schroepfer when describing a Deep Learning system based on neural networks.

He compares the approach to sealing off a child's brain when they finish school, to telling them 'You're done learning about the world'.

The fundamental limitations of such an approach is why Facebook's AI Research team has been working to add something akin to human short-term memory to its Deep Learning systems.

"It is, in my opinion was, a fundamentally missing component of AI. There's no way we can build AI systems that are able to do the sorts of things we want to do without this capability to learn and to retain new facts they've never seen."

By grafting a short-term memory onto neural networks, Facebook has already demonstrated systems capable of more complex reasoning than such learning software is typically capable of. In turn, this addition gives these systems the capacity to respond sensibly to more complex queries.

From when Facebook's prototype system first started using this memory in March, it has progressed from being able to be retain thousands of "pieces of information" to hundreds of thousands.

"It's scaling quite well," said Schroepfer.

The challenge for Facebook now is creating systems that more closely mimic the way humans store information, a task that is likely to be far more complex, he said.

"A lot of the techniques we're looking at now, that's mostly for short term memory. The human has a small short-term memory and much bigger long-term memory, so you want a closer memory architecture to that.

"We're looking at different ways of doing that but that's a much longer project," he said - describing it as one of Facebook's "10-year" bets.

Building an AI that understands the world

One of Facebook's first demonstrations of this memory networks technology was showing how a system could answers questions about the plot of a movie after being fed a synopsis of the film.

Facebook gave the system an outline of The Lord of the Rings and it was able to successfully answer questions about where characters and key items, such as the the ring of power, were at different points throughout the story.

"You can see basic reasoning and basic understanding of relationships of the ring with the person, where they are and where they were," said Schroepfer.

Playing games

For technological marvels painted as harbingers of both mankind's destruction and salvation, AIs also spend a lot of time playing games.

While Google Deepmind continues to ace humans in a growing catalogue of classic video games, at Facebook machine intelligence is focused on cracking a more ancient and complicated challenge.

The 2,500 year old game of Go is sometimes referred to as the Chinese version of chess. But unlike chess, humans still reign supreme at Go. While machines can brute force chess, running through each of the many different possible moves in a blink of eye, the myriad possibilities in Go make looking ahead in this way ineffective.

In the case of Go, Facebook's AI is effectively watching the best human players to learn how to win. After several months of a visual recognition system being fed moves from Go grandmasters the system is learning the arrangements of pieces associated with winning moves.

Schroepfer believes the approach is similar to how humans learn how to play the game and says the system is already able to beat the best amateur human players.

"In a short number of months we've built a Go AI that can beat some of the AIs that were designed specifically for the purpose of playing Go and is as good as a very good amateur human player," said Schroepfer.

"The lesson is that by combining these different technologies you could very rapidly build something that is better than things that people have been working on for many years. I think this will be one of the many ways we'll see advances in artificial intelligence in the future."

Seeing the world as people do

Of course, what Facebook really wants to use this system for is to help it better organise posts from the more than one billion people that use the social network each day.

Given the volume of content being generated, Facebook needs an automated solution and is teaching an AI to better understand what is being shown in an image.

"One of the things we've been able to do is categorise what's in an image. Knowing that this is a game of baseball and it's daytime," said Schroepfer.

Describing what's in an image is a difficult prospect for a machine. Even getting a computer to look at a picture of a man holding a bat and determine where the bat ends and the man's hand begins is tricky. However, Schroepfer said that Facebook has recently demonstrated a dramatic improvement in how to segment an image into its component parts. The social network will present these findings in a paper at a machine learning conference next month.

Image: Facebook

Getting a computer to accurately describe an image is a significant achievement. Facebook is considering whether the technology could form the basis for an automated system that describes images posted to Facebook to the visually impaired, even allowing the person to ask questions such as 'What is the man doing?' or 'Where is the baby?' and get a sensible answer. The social network has already built a prototype Android app to demonstrate this capability.

The breakthrough should also let Facebook better screen content. For instance, if the software that governs what appears in a person's News Feed understood what a photo of a soccer game looked like it could ensure that someone who hates football didn't see photos of matches posted by their soccer-loving best friend.

"All of this is about understanding the world as it exists today and better helping to filter and manage that," said Schroepfer.

The improved reasoning of the computer system has also allowed it to make accurate predictions about other aspect of the world, with Facebook training a system that can look at a stack of blocks and successfully assess whether they are likely to fall over more than 90 percent of the time.

"It's one of the many ways we're trying to help systems understand what's going to happen in the future and then help us...to plan," said Schroepfer.

The future for Facebook's AI

This technology also has the potential to help Facebook scour its vast stores of data and make more accurate inferences about who its users are.

Given that Facebook has a financial incentive to build better profiles of its users, much of its $12.47bn revenue is generated from serving ads, is the network looking to use these systems to piece together more detailed pictures of its users' lives? Schroepfer says not at present.

"I don't think this stuff is hooked up to that stuff at all. At the moment this is all prototype work, we're generally very clear with people about how we use their data, so we'll make sure that's clear for everyone."

However, in spite of these assurances, Facebook has faced repeated criticisms about how user data is made accessible to third parties, most recently over a service that allowed anyone with a person's phone number to retrieve that user's personal profile.

Facebook's progress towards building an AI that can interact with the world in a more human-like way will also be put to use in its virtual assistant, named M.

At present only a select group of users have access to M, a virtual assistant that helps people to schedule appointments and make travel arrangements.

Image: Facebook

Unlike automated services such as Apple's Siri and Microsoft's Cortana, many of the queries that are made to M are handled by humans who are "backed up by AI".

As that AI becomes more sophisticated it will gradually handle a greater share of the queries.

"Unlike other fully automated systems there's no limit to what you can ask M to do, beyond the law and reasonableness," said Schroepfer.

"This memory network stuff we've talked about is already plugged into the system," he said, adding "we've been able to get some really great results".

These results include the system responding to queries with some of the common-sense checks that a human would make. Schroepfer said that when asked to order flowers the system had now learned to enquire 'Where are you getting them delivered' and 'What is your budget?'.

"The end game here is quite simple. Having a digital assistant that can do lots of things for you is a huge empowerer and saver of time.

"When these AI systems get good enough, we could afford to scale AI to the entire planet. That's the most important thing about all of this. It's a superpower we could give to every person on the planet."