The maps that are particularly built for self-driving purposes are usually called High Definition Maps or HD Maps. These maps specifically have extremely high precision at centimeter-level. Read this article to understand everything about HD Maps.

Table of Contents

What are HD Maps? Why are HD Maps needed? How are HD Maps made? What consists of HD Maps? What is the relevance of HD Maps in a Self-driving car? How many cars do we need to create HD Maps? What are the challenges in creating HD Maps? How are HD Maps stored? What does the car really see on an HD Map? Is it important for HD Maps to be visually aesthetic? What are the legal issues of HD maps? Are maps exploding in the era of HD Maps? What will be the unexpected consequences of getting HD Maps?

What are HD Maps?

A High Definition Map(HD Map) used for autonomous vehicles.

The maps that are particularly built for self-driving purposes are usually called High Definition Maps or HD Maps for short. These maps specifically have extremely high precision at centimeter-level. This is because the robots need very precise instructions on how to maneuver themselves around the 3D space.

Why do we need such precise maps for driving down the road? In most cases, the tolerance for error might be high, but there might be cases such as driving on a road to the town hall that literally cliffs on one side, where there is no room for error. So, the maps need to be extremely precise and contain a lot of information, which humans may take for granted. Not only that the maps should contain where the lanes are, where the road boundaries are, we also want to know where the curves are and how high the curves are.

If it is 5 cm, we are approaching an era of a holy grail for mappers, as a 1:1 map if ever really made would be as big as the world itself. So, HD Mapping is really a 1:1 mapping. HD Maps are not just about scale, but also about comprehensiveness.

Why are HD Maps needed?

Use of digital maps in navigation devices and mobiles has become passé, more so, because these maps are simple, primarily meant for humans, who can understand simple instructions as they navigate. In this era of autonomous vehicles, where machines and robots need to make decisions on roads, we need a new set of maps, purposefully built for robotic systems. Maps now need to be democratized beyond humans.

While robots have the capability to do some things more efficient than humans, humans are still much wiser. The real-time decision-making capability, when it comes to driving and navigation is one of those key areas, where humans still have the edge. For example, decisions humans really take for granted such as stopping the vehicle at the right place, watching for a traffic signal at the intersection, or to avoid a split in the last minute to avoid an obstacle on the road, become very hard for robots to make. So, as part of the decision-making process, mapping becomes a really critical component of helping the robots make the right decisions at the right time.

How are HD Maps made?

At a very high level, there is a hardware component as well as a software component. A hardware component is more visible because if we look at the picture of a self-driving car, you will quickly recognize it because it has a lot of sensors typically around its rooftop. And these sensors are really useful for map creation and map update purposes. There is a combination of sensors such as cameras, LiDAR, GPS, IMU, and radars. These sensors typically also help in providing redundancy to the self-driving cars, in case if any of the sensors fails.

Various sensors in an autonomous vehicle or a self-driving car. Credits: HERE

When it comes to a self-driving car to precisely locate itself, the visual sensors such as cameras typically have limitations. This is where the LiDAR sensor plays a role in precisely measuring the depth or distance in a 3D space. However, both LiDAR and cameras typically work together, running very fast, multiple times per second. If a car is typically driving at a speed of 70 miles/hr, the sensors are collecting a lot of data at that high speed as well. In this scenario, the car is consuming and creating the maps at the same time.

The other part of this is the software. The software part is also really interesting, as the software powers the hardware to collect and record the data and this information must be made sharable.

Sophisticated sensors make autonomous cars think better Autonomous vehicles use a combination of sensors that help in capturing the data required for creating HD maps. The main types of sensors include Cameras, Long Range Radar, Short & Medium Range LiDAR and Ultrasound.

What consists of HD Maps?

According to Lyft, an HD Map is organized into five layers. They are a base map (standard definition map), geometric map, the semantic map, map priors, and real-time knowledge.

Layers in an HD Map – Credits: Lyft

Geometric Map is composed of raw sensor data collected by raw sensor data from lidar, various cameras, GPS, and IMUs. The output is a dense 3D point cloud, and this data is post-processed to produce derived map objects that are stored in the geometric map.

Semantic Map Layer is built upon the geometric map layer, by adding semantic objects. Semantic objects can be either 2D or 3D such as lane boundaries, intersections, parking spots, stop signs, traffic lights, etc. that are used for driving safely. These objects contain rich information such as traffic speeds, lane change restrictions etc.

Map priors layer contains dynamic information and human behavior data. Examples such as the order in which traffic lights change, the average wait times in a typical day at the lights, the probability of a vehicle at a parking spot, the average speeds of vehicles at parking spots etc. Autonomy algorithms commonly consume these priors in models as inputs or features and combined with other real-time information.

Real-time knowledge layer is the top-most layer in the map that is dynamically updated contains real-time traffic information. This data can also be shared in real time between the fleet of autonomous vehicles.

What is the relevance of HD Maps in self-driving cars?

On a very high level, the software that runs the self-driving car consists of a software stack with four components, the first component is the perception system; you can think of it as eyes of humans. It’s trying to see what is on the road that the car is figuring out, for instance, do I see a human crossing the road or do I see a signal turning from red to green?

Another piece is called localization – and localization module tells the car where you are in the 3D space, and what’s actually around you. Example, the system says that you are 150 centimeters from the next stop line, here is a cross-walk of this width etc.

That’s when the Planning & Control module will kick-in and say, “Ok I am going to slow down, and then going to make a full stop at the next intersection.”

And then the fourth component is the mapping component, and you can imagine the mapping system to having tentacles above the three components, just mentioned earlier. For instance, mapping and localization work, very tightly together, constantly comparing where you are on the map. It needs to know which object was supposed to be here and tell the difference. So, if I know there is supposed to be an intersection and a crosswalk, and I see a moving object crossing it, that’s probably a pedestrian.

How many cars do we need to create HD Maps?

A single car can’t really map the entire world. The question is not only how the maps are created, but how a map is maintained to reflect the changes on the road. For this, multiple cars would need to drive on the same road, and that data would be aggregated from multiple drives on multiple cars together. The more the number of cars you have on the road, the more data gets collected, and the higher will be the quality of the HD maps.

An autonomous car sharing real-time traffic information with other cars

What are the challenges in creating HD Maps?

The cost incurred in creating the HD Maps is a big challenge, as we cannot send huge amounts of data over cellular networks. The way to address this challenge is to categorize the information to be shared in real-time vs the information that need not be shared in real time. So, things that really impact driving behavior needs to be shared and distributed with other cars that may be affected such as a known accident on the road, landslides during a rainy season, a road construction in progress etc.

How are HD Maps stored?

There is a lot of data being collected now and there will be more data that will be collected in the future once we have more self-driving cars on the road. This is where the Cloud infrastructure comes into place, both for storage and computation purposes, and 99% of the computation is done over there.

However, each individual car will continue to have its own memory cells or storage that people call as edge computing. So, each car will carry its own storage and computing power, so that each car can make its own decisions independently, even if it is offline it should be fully functional. In that case, the maps are being continuously called by the rest of the software stack such as the perception system, planning and control system many times per second. The change detection module needs to kick-in to check what looks different from the map the system already has, and identify if it is a really important change to be distributed in real-time or send it to the edge compute process, and activate the active data collection process and share that data within the cloud.

What does the car really see on an HD Map?

How we want to visualize depends on how we ultimately want to consume it. Unlike humans who need to see to visualize the data, the robots need to consume the data, which they do through APIs and so on.

REST API for impeccable training data of Self-driving cars – Scale Inc.

So, why is there a need to have a visual representation at all then? The answer is; those are for humans! As part of the map creation and update process, we need to have human operators to check the computer output. To see, the ambiguities in the real world, for example, a complex intersection – that makes it even impossible for humans some time to interpret the traffic rules. So, we have these visualization tools that really help in the map creation and maintenance process.

What does a self-driving vehicle see? Credits: Scale Inc.

Is it important for HD Maps to be visually aesthetic?

When it comes to the creation and maintenance of HD Maps, functionality triumphs over the beauty aspect, as it is important to ensure the quality of the map is really high. Otherwise, the human operators won’t be able to work effectively. So, productivity and efficiency are above all than the design.

But when these maps are displayed for other purposes for simulation and virtual reality, then depending on the use case, the design choices of the maps are made.

What are the legal issues of HD maps?

Maps have always carried this aspect of law of the land, issues with geographic names, legal issues such as where someone is allowed, where someone is not, where the maps can be shown, where they cannot be.

Even though digital maps are widely available today, there are countries around the world that heavily regulate their geospatial data, and even forbid exporting geospatial data. This problem is inherent though from normal navigation maps.

When it comes to HD Maps, this is going to be a huge challenge, as the government regulations are not clear. There is going to be all sorts of sensitivities, as the cars have very high-resolution LiDAR and then the cameras are constantly scanning the streets and seeing areas like private driveways etc. So, being able to protect people’s privacy and also follow government regulations from a security point of view, and ensure that data is sufficiently encrypted so that it won’t accidentally leak out are some key areas to be addressed.

In some countries, the governments are getting active now where they are trying to both protect privacy aspects and also advance the self-driving technology. It will be a few years before the dust settles and we see an action coming from technology and policy standpoint.

Aspects related to who owns the data are also to be addressed – is it the car owner who owns the data or the car maker or should the government own the data? These aspects were always there with maps and will be getting more powerful with the advent of dynamic maps.

How can we say that the maps are exploding in the era of HD Maps?

All these years, we tend to have simplified the maps to the very extent possible. To the geographic boundaries, to the oceans to the roads. But now, we are moving in a different direction where the maps are simply exploding and scale and dimension and maps have become reality.

And the real lag between consuming and creating a map is reducing, cars will have the capability to share information, which was not possible in case of mapping ever. Mapping will now become more of an organic process, with a continuous collection capability in place.

What will be the unexpected consequences of getting HD Maps?

When we are creating HD Maps of a city, we are creating the digital infrastructure of a city’s road network. And every city authority tries to do the same, as most are now having the GIS data for the city – which are created by sending out the surveyors. Public works departments often do the same. These are mostly static, very labor intensive, very costly to do and these kinds of surveys can’t be repeated all the time.

On the other hand, when we have cars with the right sensors running on the road, they are constantly updating the road. So, the process where a central authority from the government maintains this digital infrastructure can suddenly be solved by people with cars or just with cars. And this information is really useful just beyond self-driving.

As everybody becomes a mapper, everybody contributes when they are driving, and thus people become an integral part of this knowledge creation, and it becomes a means of empowering people.

Credits: a16z Podcast: Exploding the Map with Wei Luo, David Rumsey, and Hanne Tidnam