Keyhole — The Google Earth Predecessor

Like many other famous Google products, the core technology for Earth did not originate at the company. Intrinsic Graphics developed it in the 1990s. But Intrinsic was primarily focused on gaming, and the company board created a subsidiary called Keyhole to pursue the map project. Its clients were companies in real estate, urban planning, defense, and intelligence.

Ultimately, Keyhole could not make enough money to cover the high costs of procuring satellite imagery. But the struggling company saw a turnaround in 2003 when its imagery and logo were extensively used by news media to report on the invasion of Iraq. Around the same time, Google observed that a good chunk of its searches involved maps and directions. And thus, in 2004, Google acquired Keyhole. But Jerry Brotton, who covers the story of Google Earth in his book A History of the World in Twelve Maps, told the Independent, “Larry Page and Sergey Brin just thought it was cool. I don’t think they realized what they’d got.”

Behind the scenes — Google Earth imagery

Google Earth relies on satellite and aerial imagery for maps and imposes other data on top of this.

Gif from video by Nat & Friends

Google Earth stitches together tens of millions of images to create composite imagery at varying distances. The Landsat and Sentinel satellites provide the primary images. The Landsat 8, which was launched by NASA and the United States Geological Survey, provides imagery with a resolution at 30 meters captured once every two weeks. Sentinel satellites are a part of Copernicus, the EU’s earth observation program. In addition to these sources, Google procures data from other agencies around the world. This data is commercially available, but Google Earth combines them and makes them easily accessible in a way no one provider can. As you zoom in, you see a different set of composite images that offer finer detail. For this, Google relies mainly on the imagery provided by the private company DigitalGlobe and its WorldView satellite, which is equipped with cameras that can capture from space objects that are merely 25 cm apart on the Earth’s surface.

Since the imagery comes from a variety of sources, they have varying resolutions and timestamps for different areas and zoom levels. The satellites can only make a limited number of orbits per day and given the massive demand for their cameras from various agencies and companies, some regions are given preference and have crisp images at street level, while others appear pixelated.

As you zoom in closer to the surface, Google starts using imagery from aerial photography. These images are collected using special planes fitted with multiple cameras. These planes fly zig-zag over a selected region to get multiple high-quality, overlapping images, from all angles. By combining multiple aerial photos in a process called photogrammetry, and using the elevation and topography information provided by NASA’s Shuttle Radar Topography Mission, Google Earth enables 3D view. Photogrammetry involves creating a mesh using depth maps from the various cameras and adding the texture of individual structures to this mesh. A team at Google put together the video below explaining the imagery process with insight from the Google Earth team.

This team updates the entire globe image once every couple of years, and as you zoom in, the images are updated on a more frequent basis. For some cities, Google updates the images multiple times a year. Google Earth also gives users a time slider to see previous versions of the earth through a feature called Historical Imagery.

The secret sauce of Google Earth

The key technology of Google Earth is not the imagery or the process of combining it. Instead, it’s the tech behind delivering all this quickly and efficiently to our devices.

By breaking up the high-resolution imagery into much smaller chunks and serving them to the user only when needed, Google Earth can efficiently deliver the world to people’s hands.

A whole Earth image in which one pixel represents every square kilometer of earth’s surface would be 40,000 pixels wide and 20,000 pixels tall. This image would be at least 2.4 GB in size, enough to choke the graphics card of an average computer. And even that image wouldn’t be detailed enough to show buildings, rivers, or other essential landmarks. The high-resolution Google Earth that we are accustomed to would need 16 million times more storage. But Google Earth faces none of the challenges described here because of its patented technology that uses two features called mip-mapping and clip-stacking.

As the user zooms out, the high-resolution surface details are indiscernible to the human eye. Mip-mapping takes advantage of this fact and swaps out high-resolution images for less-detailed variants as the user zooms out. Clip stacking places these mipmaps in a way that only the current area viewed by the user gets drawn. When the user moves the map around, the app streams details of the new focal point and its surroundings. By breaking up the multi-terabyte, high-resolution imagery into much smaller chunks and serving them to the user only when needed, Google Earth can overcome user device constraints and efficiently deliver the world to people’s hands. If you want to dive into the nitty-gritty details of this, check out this blog post by Avi Bar-Zeev, who co-founded Google Earth’s predecessor, Keyhole.

Google Earth = maps + data

In addition to imagery, Google Earth superimposes various other layers of geospatial information on the map. This includes surface temperature, weather data, terrain, land cover, water cover, bathymetry, cropland, population, income, and other types of geospatial data. Google Earth collects all this data from thousands of third-party sources and gives researchers, and the general public alike, an invaluable study tool. The team at Google, along with its partners like National Geographic and Sesame Street, also bring educational initiatives through a feature called Voyager. This feature lets users tell stories and teach new concepts using interactive games and cultural tours that take people around the world.

With Google Earth, you can wander the planet, learn more about the world we live in, or conduct groundbreaking research. But we all know what you’ll most likely do first: search for your house. But don’t stop there. You can get creative and use Google Earth to plan a 4,450-mile route that spells “Marry Me” to propose to your loved one. Or you can use it to look for hidden treasure, like this man in England did. And he did, in fact, find 1.5 million pounds worth of Saxon coins. If you are tired of exploring Earth, then worry not. Google Earth also lets you switch views and explore the night sky and all its stars, or enter Mars mode to see the red planet or the Moon mode to explore earth’s only natural satellite. As Google Earth product manager Gopal Shah once said, “Google Maps is for finding your way, Google Earth is for getting lost.”