Prediction

When we drive a car today, we’re subconsciously estimating the next few seconds of behavior from other road users — anticipating when a pedestrian might jaywalk or when another car may be about to cut us off. Attentive drivers are incredibly good at reacting in these situations — managing their speed and planning out contingencies to adapt to anomalous behavior from others. These same actions, which good drivers perform quickly while avoiding a drastic response, are also required for a self-driving car to navigate busy city streets.

We must build algorithms that enable our autonomous vehicle to respond to a deeper understanding of the likely behavior of other road users. We need to instill “thoughtfulness” into the technology to ensure that the car can operate safely, reliably and predictably.

For example, the car needs to know when it will have to move over slightly for a large truck to give it more room, or adjust its speed to stay out of another driver’s blind spot. At the same time, we have to build algorithms that enable it to know when it’s being overly conservative, when the car will need to “nudge” in dense traffic, or commit to an action consistently so that other road users can respond correctly. Throughout, as the computer absorbs all of the information, it’s key that it never gets distracted or learns the wrong model, as it will act strangely in anticipation of an action that never comes to fruition.

This is the balance we must deliver in building these predictive models, and it’s only from all of these examples and real-world driving that we can learn to predict the micro-maneuvers that turn out to be the leading indicators of the likely actions of other road users.

System integration and testing

Generally, the software that powers a self-driving car is what’s called a stochastic system. What this means is that the results are determined through a series of detected patterns and models applied to inherently random sensor inputs, rather than through a mathematical equation with a consistent set of inputs that translates to a consistent set of outputs.

Imagine driving down the same road twice and nothing changes between the first and the second trip. You’re highly unlikely to drive the same path at exactly the same speed the second time around. Self-driving vehicles are no different, though in general they will be more consistent than human drivers.

Testing stochastic systems requires a significant number of repetitions generated by real-world data for it to be representative. That means we must gather millions of miles of road experience to teach the software to drive with confidence. (Imagine needing to drive millions of miles to get your driver’s license!) But not all miles are created equal, so “accumulated miles” is not an expressive enough metric to track progress. Think of it this way: The skills you acquired learning to drive in a quiet Midwestern town will not translate should you find yourself driving in the heart of Manhattan.

The algorithms we build move millions of pixels per second through complex math and logic to calculate important outcomes about the state of the world of an autonomous vehicle. Given the high dimensionality of these inputs, it’s impossible to test across the space of every possible input combination — there would be many trillions of combinations to test, which is simply unmanageable.

So we need to be clever about how we use the recorded miles of driving experience from our test vehicles. We’re building tools that can extract the right set of miles that sufficiently covers the realistic and relevant scenarios that the vehicle is likely to see, and then test for the right response. This balance requires extensive driving experience and data collection in the target deployment area that covers as many diverse and challenging scenarios as possible. We must also collect sufficient variations around environmental changes in each scenario that might degrade a sensor’s output, such as weather and lighting conditions.

We have built a dedicated team of logistics and test operators to safely execute these test miles, plus a team of analytics professionals and software engineers who are creating the tools to manage the data flow that gives us confidence in our completeness in scenario coverage.

We’re still very much in the early days of making self-driving cars a reality. Those who think fully self-driving vehicles will be ubiquitous on city streets months from now or even in a few years are not well connected to the state of the art or committed to the safe deployment of the technology. For those of us who have been working on the technology for a long time, we’re going to tell you the issue is still really hard, as the systems are as complex as ever.

Everyone knows focused teams that innovate and work hard can solve amazingly difficult problems. At Argo, we see these challenges as an inspiration. They drive us to leverage the advancements of the last decade to propel us into a new era where the commercial success of self-driving cars will be a reality.

We’re taking a pragmatic approach to bringing about fully self-driving cars — incorporating the state of the art while acknowledging there’s no silver bullet. We’re playing the long game and avoiding the hype in our commitment to bring this important technology to maturity in the form of a great product that earns the trust of millions of people around the world.