It's no secret that I've been a critic of the NHTSA "levels" as a taxonomy for types of Robocars since the start. Recent changes in their use calls for some new analysis that concludes that only one of the levels is actually interesting, and only tells part of the story at that. As such, they have become even less useful as a taxonomy. Levels 2 and 3 are unsafe, and Level 5 is remote future technology. Level 4 is the only interesting one and there is thus no taxonomy.

Unfortunately, they have just been encoded into law, which is very much the wrong direction.

NHTSA and SAE both created a similar set of levels, and they were so similar that NHTSA declared they would just defer to the SAE's system. Nothing wrong with that, but the core flaws are not addressed by this. Far better, their regulations declared that the levels were just part of the story, and they put extra emphasis on what they called the "operating domain" -- namely what locations, road types and road conditions the vehicle operates in.

The levels focus entirely on the question of how much human supervision a vehicle needs. This is an important issue, but the levels treated it like the only issue, and it may not even be the most important. My other main criticism was that the levels, by being numbered, imply a progression for the technology. That progression is far from certain and in fact almost certainly wrong. SAE updated its levels to say that they are not intended to imply a progression, but as long as they are numbers this is how people read them.

Today I will go further. All but level 4 are uninteresting. Some may never exist, or exist only temporarily. They will be at best footnotes of history, not core elements of a taxonomy.

Level 4 is what I would call a vehicle capable of "unmanned" operation -- driving with nobody inside. This enables most of the interesting applications of robocars.

Here's why the other levels are less interesting:

Levels 0 and 1 -- Manual or ADAS-improved

Levels 0 and 1 refer to existing technology. We don't really need new terms for our old cars. Level 2 perhaps best described as a more advanced version of level 1 and that transition has already taken place.

Level 2 -- Supervised Autopilot

Supervised autopilots are real. This is what Tesla sells, and many others have similar offerings. They are working in one of two ways. The first is the intended way, with full time supervision. This is little more than a more advanced cruise control, and may not even be as relaxing.

The second way is what we've seen happen with Tesla -- a car that needs supervision, but is so good at driving that supervisors get complacent and stop supervising. They want a full self-driving car but don't have it, so they pretend they do. Many are now saying that this makes the idea of supervised autopilot too dangerous to deploy. The better you make it, the more likely it can lull people into bad activity.

Update: One day after I wrote this, it was revealed that NHTSA shut down comma.ai's efforts to build an aftermarket autopilot citing these concerns, among others.

Level 3 -- Standby driver

This level is really a variation of Level 4, but the vehicle needs the ability to call upon a driver who is not paying attention and get them to take control with 10 to 60 seconds of advance warning. Many people don't think this can be done safely. When Google experimented with it in 2013, they concluded it was not safe, and decided to take the steering wheel entirely out of their experimental vehicles.

Even if Level 3 is a real thing, it will be short lived as people seek an unmanned capable vehicle. And Level 4 vehicles will offer controls for special use, even if they don't permit a transition while moving.

Level 5 -- Drive absolutely everywhere

SAE, unlike NHTSA's first proposal, did want to make it clear that an unmanned capable (Level 4) vehicle would only operate in certain places or situations. So they added level 5 to make it clear that level 4 was limited in domain. That's good, but the reality is that a vehicle that can truly drive everywhere is not on anybody's plan. It probably requires AI that matches human beings.

Consider this situation in which I've been driven. In the African bush on a game safari, we spot a leopard crossing the road. So the guide drives the car off-road (on private land) running over young trees, over rocks, down into wet and dry streambeds to follow the leopard. Great fun, but this is unlikely to be an ability there is ever market demand to develop. Likewise, there are lots of small off-road tracks that are used by only one person. There is no economic incentive for a company to solve this problem any time soon.

Someday we might see cars that can do these things under the high-level control a human, but they are not going to do them on their own, unmanned. As such SAE level 5 is academic, and serves only to remind us that level 4 does not mean everywhere.

Levels vs. Cul-de-sacs

The levels are not a progression. I will contend in fact that even to the extent that levels 2, 3/4 and 5 exist, they are quite probably entirely different technologies.

Level 2 is being done with ADAS technologies. They are designed to have a driver in the loop. Their designs in many case do not have a path to the reliability level needed for unmanned, which is orders of magnitude higher. It is not just a difference of degree, it is one of kind.

Level 3 is related to level 4, in particular because a level 3 car is expected to be able to handle non-response from its driver, and safely stop or pull off the road. It can be viewed as a sucky version of a level 4 system. (It's also not that different -- see below.)

Level 5, as indicated, probably requires technologies that are more like artificial general intelligence than they are like a driving system.

As such the levels are not levels. There is no path between any of the levels and the one above it, except in the case of 3/4.

Level 4

This leaves Level 4 as the only one worth working on long term, the only one with talking about. The others are just there to create a contrast. NHTSA realizes this and gave the name ODD (Operational Design Domain) to refer to the real area of research, namely what roads and situations the vehicles can handle.

The distinction between 4 and 3 is also not as big as you might expect. Google removed the steering wheel from their prototype to set a high bar for themselves, but they actually left one in for use in testing and development. In reality, even the future's unmanned cars will feature some way in which a human can control them, for use during breakdowns, special situations, and moving the cars outside of their service areas (operational domains.) Even if the transition from autodrive to human drive is unsafe at speed, it will still be safe if the car pulls over and activates the controls for a licenced driver.

As such, the only distinction of a "level 3" car is it hopes to be able to do that transition while moving, on short but not urgent notice. A pretty minor distinction to be a core element of a taxonomy.

If Level 4 is the only interesting one, my recommendation is to drop the levels from our taxonomy, and focus the taxonomy instead on the classes of roads and conditions the vehicle can handle. It can be a given that outside of those operating domains, other forms of operation might be used, but that does not bear much on the actual problem.

I say we just identify a vehicle capable of unmanned or unsupervised operation as a self-driving car or robocar, and then get to work on the real taxonomy of problems.