A decade later, Street View cars have snapped more than 80 billion photos in thousands of cities and 85 countries. The company’s conventional mapping data is even more extensive. But Google still hungers for a better index of the world. Jen Fitzpatrick, the vice president who heads the company’s maps division, blames that on us. “People are coming to us every day with harder and deeper questions,” she says.

The first time you searched Google Maps or Street View you probably typed in a street address—perhaps your own. Fitzpatrick says the company now gets tougher queries that require a fresher, more detailed digital model of the world, like “What’s a Thai place open now that does delivery to my address?”

She wants her service to handle queries that assume knowledge of what the world looks like: “What’s the name of the pink store next to the church on the corner?” Google’s push to get us talking with its Siri-style virtual assistant encourages us to be more conversational in our demands. “These are questions we can only answer if we have richer and deeper information,” Fitzpatrick says.

Google’s huge investment in machine learning and AI provides a natural way to get that information. Thanks to recent research inside the maps division, when a Street View car captures photos of a stretch of road, algorithms can now automatically create new addresses in the company’s maps database by locating and transcribing any street names and numbers. Street View was the first of Google's product groups to use the company's powerful custom AI chips, dubbed TPUs.

The team's system has learned to figure out abbreviations, such as “AV.” for avenida, by taking hints from other signs in the country where they’re spelled out in full, and other clues in Google's maps data. Software has also been trained to recognize business names, and is smart enough to ignore visual trip hazards like the giant Bridgestone logo that might dwarf the name of a tire shop.

Higher quality images coming from the new hardware now atop Google’s Street View vehicles will allow those systems to extract information like that more reliably. “From a machine learning perspective, everything gets better,” says Andrew Lookingbill, an engineer working on the technology. It will also help his team’s efforts to build new software even better at understanding the world. They’re thinking about trying to automatically recognize different types of business from their appearance and reading finer-grained information like opening hours signs.

Google

New territory

Decoding Street View imagery with algorithms can be especially useful in places where roads, cities, and businesses are changing fastest—the less-developed economies where Google and its competitors hope to find their next few billion users. The government of India reported this year that it has recently laid an average of 14 miles of new road every day. Street View went live this summer in Nigerian megacity Lagos—population 21 million. Fitzpatrick says that Google’s image-scouring algorithms could help translate the new imagery into a significant bump in map quality. Google sells ads inside maps, so new coverage and accuracy can translate into more revenue if they draw new users and usage to the service.