When Adrian Ludwig describes the ideal approach to computer security, he pulls out an analogy. But it's not a lock or a firewall or a moat around a castle. Computer security, he says, should work like the credit card business.

A credit card company, he explains, doesn't eliminate risk. It manages risk, using data describing the market as a whole to build a different risk profile (and a different interest rate) for each individual. Computer security, Ludwig believes, should work in much the same way. "The model of good and bad—white and black—that the security community prescribes?" he says. "It's going to be all black unless we accept that there are going to be shades of gray."

If the future of security lies in managing risk, as Ludwig believes, then the future of security is machine learning.

This is pretty much what you'd expect him to say. Ludwig works at Google, where he oversees security for Android, a mobile operating system that always included as many phone makers, apps, and people as possible. But he and his colleagues aim to take this idea in a new direction. If the future of security lies in managing risk, he explains, then the future of security is machine learning, the same breed of artificial intelligence that has proven so successful in so many other parts of the Google empire. We shouldn't code hard-and-fast digital rules that aim to stop all online attacks. As the internet grows more complex—as it reaches more people—this would end up shutting everyone out. Instead, we should build systems that can analyze the larger landscape and learn to identify potential problems on the fly.

With his comparison to credit card companies, Ludwig is separating Google from Apple, its main rival, the company that so tightly controls the iPhone. "I don't want the solution to be: 'We close everything off,'" Ludwig tells me. Needless to say, the Apple security model does have its advantages. The Federal Communications Commission is investigating why it takes so long to plug security holes on Android phones—a problem that's likely the result of a fragmented Android system that has Google working with so many different phone makers. Apple just works with one phone maker: itself. But Ludwig's point is that there can be a happy middle ground between laissez-faire and lockdown. And that involves machine learning, including an increasingly important AI technology called deep neural networks.

Justin Kaneps for WIRED

"If you have a billion devices that are out there—no matter how good your security is—some of them are going to have bugs, some of them are compromised," says Ludwig, who spent eight years inside the National Security Agency and a few more with @stake, a security consultancy, before joining Google. "To manage that, you need data, and you need to analyze it."

A Deep Instinct

He's not the only one pushing this big idea. Baidu, "the Google of China," uses deep neural networks to identify malware. So do security startups such as Deep Instinct and Cylance. Just as a neural net can identify the particular characteristics of a photo, it can recognize a malicious software application—or a bit of flawed operating system code that exposes your phone to malicious hackers.

But the revolution might not be here just yet. Google's effort is still in the early stages. "It's not a science experiment. It's real. But it's not the dominant solution," Ludwig says. At the moment, Google doesn't have the volume of problems it needs to train its neural networks as completely as it would like. "Most apps are safe and good. And there's a few bad players," says Rich Cannings, who works alongside Ludwig. "It's really hard to find that bucket." Ironically, to really embrace machine learning, Google needs more Android problems to feed the neural network—or better neural networks.

That's not to say that Android's security record is spotless. "A year ago," says Joshua Drake, a researcher with a security outfit called Zimperium who recently identified a significant string bugs in Android, "I really felt that Android wasn't investing in security whatsoever." And machine learning is no cure-all. It won't help Google distribute security patches across all those Android phone makers. But it can help identify security holes—if current techniques are perfected.

Justin Kaneps for WIRED

Bouncer at the Door

Sebastian Porst runs the Google team charged with identifying any malicious or vulnerable applications that might show up on an Android phone. And he wants to put himself out of a job. Ultimately, he wants machines to do the work. "That's the goal," he says.

At Google, this is hardly an unusual attitude. In fact, it's the philosophy that drives so much of the way the company operates. "We end up with a team of people who will quickly become bored by performing tasks by hand and have the skill set necessary to write software to replace their previously manual work," says Ben Treynor Sloss, who oversees the Googlers charged with keeping its myriad online services up and running.

Inside the Android security team, this effort isn't quite as far along, but Porst and his team have built an automated system that moves things at least partly down the same road. Dubbed Bouncer, this system analyzes every app uploaded to the Google Play Store, looking for malicious or otherwise problematic software code, and then it runs each app, so it can analyze behavior as well. It also ties into the Google web crawler—the tool that indexes the Internet for the company's search engine—so it can automatically scan Android apps uploaded to random websites. "We scan apps from every source we can get our hands on," Porst says. If an unknown app is dowloaded to a certain number of Android phones, the system will grab it and analyze its code and behavior too.

Justin Kaneps for WIRED

In the past, Bouncer operated according to a predefined set of rules. But now, in an effort to hone the system, Google also leans on machine learning. In scanning all those apps, the system has gathered enormous amounts of data about each one, which Porst calls "signals"—characteristics and behavior that define the app. Now, the team is feeding these signals into neural networks so the system can learn which combinations of characteristics indicate malware. "We can use machine learning to figure out which of these signals are actually correlated with potentially harmful behavior and which are completely harmless," Porst says.

It works. But only up to a point. At the moment, Porst says, "security expertise cannot be replaced by any machine learning algorithm." Indeed, machine learning is just one part of the team's scanning pipeline, and if the system flags an app as problematic, the human engineer always double-checks its work. The trouble is too little data. Porst says the techniques are much more effective for apps outside the Play Store than those inside, mainly because, nowadays, there's almost no harmful software uploaded to the store. Nowadays, miscreants pretty much know not to try.

Which is not to say the machine learning won't improve for Porst and team. Eli David, chief technology officer at security startup Deep Instinct, says his company has built effective models by analyzing data not just across one computer platform—say, Android—but all platforms. "Your scope," he says, "must be large."

The Real Clusterfuzz

Jon Larimer landed a job at Google after identifying a gaping hole in the mobile OS. He found a flaw in a graphics device driver, the code that renders graphics on Android phones. He wrote an exploit that used this flaw to gain complete control of phones via the Internet. And then he showed it to Ludwig and crew. "You gotta meet people somehow," Ludwig says.

Now, Larimer and his team are building a system that can identify such holes on its own.

Justin Kaneps for WIRED

Android security engineers like Nick Kralevich build code that aims to eliminate exploitable holes in the OS. But holes still turn up. So, drawing on technology originally built by the team that handles security for Google's Chrome web browser, Larimer and others are building a system that deals in fuzz testing, which seeks holes in software by throwing it all sorts of random inputs. Fuzzing is a common thing, but this system—known as Clusterfuzz—simultaneously fuzzes dozens upon dozens of Android phones.

In some cases, it tests virtual incarnations of these phones, analyzing about 1,500 across thousands of servers at any given time. It also tests physical devices, because the hardware can really change a phone's behavior. Inside Building 43 at Google HQ, you'll find massive racks where hundreds of phones plug into this sweeping system. "We don't have a lot of people on our team," he says. "But the edge that we have is scale, access to thousands and thousands of CPUs."

Now, in an effort to identify more bugs, Google is applying machine learning to the problem. Larimer and team are exploring neural nets that can recognize the structure of each file encountered by Clusterfuzz. If the system knows the structure, it can test the file more thoroughly. Rather than just randomly throwing inputs at the file, it can use those that suit its particular makeup. In learning to identify how the file operates, Larimer explains, neural nets can help the fuzzer "touch as much code as possible." Like Porst's work with machine learning, the project is still young. But there's promise. "We can eventually get to the point where we can have 100 percent coverage," Larimer says. "This is where the future is."

Justin Kaneps for WIRED

Finding the Middle Ground

If nothing else, all this work shows that Android security is changing. Alongside the move towards machine learning, the company has rolled out a big bounty program, under the direction of ex-Microsoft-man Scott Roberts, and Ludwig has made a greater effort to explain how his team tackles security. Outside researchers like Joshua Drake used to question how seriously Google approached security in the early years of Android. But even Drake will tell you that he sees signs of change, particularly since he exposed the Stagefright bug last summer. "There's been a huge difference," Drake says. "It got to the point where they realized they needed to do more."

Google doesn't believe in the Apple model. But Ludwig and his Android team know that older approaches didn't necessarily work, either. They believe the ideal approach is somewhere in between. And others agree. "Both ecosystem have advantages and drawbacks," Drake says. "It's not so simple." And if that's the case, machine learning can indeed play a major role in the future of mobile security. If they can get it to work.