Would each program be limited — like one that plays lots of card games — or are you thinking of one massive system that learns how to do everything?

Eventually something more general. The idea for our research program is to slowly widen and widen those domains. We have a prototype of this — the human brain. We can tie our shoelaces, we can ride cycles and we can do physics with the same architecture. So we know this is possible.

Tell me about the two companies, both out of Oxford University, that you just bought.

These Oxford guys are amazingly talented groups of professors. One team [formerly Dark Blue Labs] will focus on natural language understanding, using deep neural networks to do that. So rather than the old kind of logic techniques for NLP, we’re using deep networks and word embeddings and so on. That’s led by Phil Blunsom. We’re interested in eventually having language embedded into our systems so we can actually converse. At the moment they are obviously prelinguistic—there is no language capability in there. So we’ll see all of those things marrying up. And the second group, Vision Factory, is led by Andrew Zisserman, a world famous computer vision guy.

But all of this research would all eventually be part of the same engine.

Yeah. Eventually all of those things become part of one bigger system.

What products at Google is your team looking to improve?

We still feel quite new to Google, but there’s tons of things we could apply parts of our technology to. We’re looking at various aspects of search. We’re looking at stuff like YouTube recommendations. We’re thinking about making Google Now better in terms of how well it understands you as an assistant and actually understands more about what you’re trying to do. We’re looking at self-driving cars and maybe helping out with that.

When will we see this happening?

In six months to a year’s time we’ll start seeing some aspects of what we’re doing embedded in Google Plus, natural language and maybe some recommendation systems.

How about video search?

That’s another big thing—do you want to type in actions like someone kicking a ball or smoking or something like that? The Vision group is working on those kinds of questions. Action recognition, not just image recognition.

What do you hope to do for Google in the long run?

I’m really excited about the potential for general AI. Things like AI-assisted science. In science, almost all the areas we would like to make more advances in—disease, climate, energy, you could even include macroeconomics— are all questions of massive information, almost ridiculous amounts. How can human scientists navigate and find the insights in all of that data? It’s very hard not just for a single scientist, but even a team of very smart scientists. We’re going to need machine learning and artificial intelligence to help us find insights and breakthroughs in those areas, so we actually really understand what these incredibly complex systems are doing. I hope we will be linking into various efforts at Google that are looking at these things, like Calico or Life Sciences.

What did you think of the movie Her?

I loved it aesthetically. It’s in some ways a positive take on what AI might become and it had interesting ideas about emotions and other things in computers. I do think it’s sort of unrealistic, in that there was this very powerful AI out there but it was stuck on your phone and just doing fairly everyday things. Whereas it should have been revolutionizing science and…there wasn’t any evidence of anything else going on in the world that was very different, right?

You’ve had successful experiments, but how difficult is it to build those into a system that hundreds of millions of people will use?

It’s a multi-step process. You start with the research question and find that answer. Then we do some major neuroscience and then we look at it in machine learning and we implement a practical system that can play Atari really well and then that’s ready to scale. Here at Deep Mind about three quarters of the team is research but one quarter is applied. That team is the interface between the research that gets done here and the rest of Google’s products.

You had a fantastic career in the gaming world and you left it because you felt you had to learn about the brain.

Yeah. Actually my whole career, including my games career, has been leading up to the AI company. Even in my early teens I decided that AI was going to be the most interesting to work on and the most important thing to work on.

But you were at the top of the game world — you worked on huge hits like Black and White and founded Elixir Studios — and you just thought, “OK, time to study neuroscience?”

It was more like, “Let’s see how far I can push AI under the guise of games. So Black & White was probably the pinnacle of that, then it was Theme Park and Republic and these other things that we tried to write. And then around 2004–2005, I felt we’d pushed AI as far as it could go within the constraints of the very tight commercial environment of games. And I could see that games were going to go more towards simpler games and mobile — as they have done— and so actually there would be less chance to work on a big AI project within a game project. So then I started thinking about Deep Mind — this is 2004 — but I realized that we still didn’t have enough of the components to make fast progress. Deep Learning hadn’t appeared at that point. Computing power wasn’t powerful enough. So I looked at which field should I do my PhD in and thought it would be better to do it in neuroscience than in AI, because I wanted to learn about a whole new set of ideas and I already knew world-class AI people.

In your years of studying the brain, what was the biggest takeaway as you started an AI company?

Lots of things. One is reinforcement learning. Why do we believe that that’s an important core component? One thing we do here is look into neuroscience inspiration for new algorithms and also validation of existing algorithms. Well it turns out in the late ‘90s, Peter Dayan and colleagues were as involved in an experiment using monkeys, which showed that their neurons were really doing reinforcement learning when they were learning about things. Therefore it’s not crazy to think that that could be a component of an overall AI system. When you’re in the dark moments of trying to get something working, it’s useful to have that additional information—to say, “We’re not mad, this will really work, we know this works—we just need to try harder.” And the other thing is the hippocampus. That’s the brain area I studied, and it’s the most fascinating.

Why?

Deep Learning is about essentially [mimicking the] cortex. But the hippocampus is another critical part of the brain and it’s built very differently, a much older structure. If you knock it out, you don’t have memories. So I was fascinated how this all works together. There’s consolidation [between the cortex and the hippocampus] at times like when you’re sleeping. Memories you’ve recorded during the day get replayed orders of magnitude faster back to the rest of the brain. We used this idea of memory replay in our Atari agent. We replayed trajectories of experiences that the agent had had during the training phase and it got the chance to see that hundreds and hundreds and hundreds times again, so it could get really good at that particular bit.

When you talk about the algorithms of the brain, is that strictly in the metaphoric sense or are you talking something more literal?

It’s more literal. But we’re not going build specifically an artificial hippocampus. You want to say, what are the principles of that? [We’re ultimately interested in the] functionality of intelligence, not specifically the exact details of the specific prototype that we have. But it’s a mistake also to ignore the brain, which a lot of machine learning people do. There are hugely important insights and general principles that you can use in your algorithms.

Because we don’t fully understand the brain, it seems difficult to take this approach all the way. Do you think there’s something that’s “wet” that you can’t do in silicon?

I looked at this very carefully for a while during my PhD and before that just to check where this line should be drawn. [Roger] Penrose has quantum consciousness [which postulates there are quantum effects in the mind that computers can’t emulate]. Beautiful story, right? You wish it’s sort of true, right? But it all collapses. There doesn’t seem to be any evidence. Very top biologists have looked carefully for quantum effects in the brain and there just didn’t seem to be any. As far as we know it’s just a classical computation device.