MADISON, Wis. — As more autonomous vehicles have hit the streets this year, more and more people are paying attention to robocars. For sure, they’re no longer a novelty. But those AVs, during public test driving, also exposed some fundamental weaknesses. Most significantly, despite their ability to hew exactly to the rules of the road, robocars have shown little aptitude for reading the human drivers with whom they will be sharing the road. Or, as Mike Demler, senior analyst at the Linley Group, put it in his rhetorical question: “How do you program a robot to have common sense?” Needless to say, in 2017, we’ve seen a few well-publicized, but non-fatal robocar accidents. A 2016 consumer study conducted MIT AgeLab and the New England Motor Press Association, shows that of about 3,000 people asked about their interest in self-driving cars, nearly half — 48 percent — said they would never purchase a car that completely drives itself. Respondents said they’re uncomfortable with the loss of control and don't trust the technology. They also don’t feel self-driving cars are safe. In short, most non-engineering consumers are skeptical or distrust autonomous technology. However, tech and auto companies aren’t fazed. They appear unconcerned about consumer adoption and don’t even expect consumers to buy a lot of autonomous vehicles — at least not soon. Automakers have decided to put off dealing with thorny “machine vs. human” consumer trust issues. It's about fleets, stupid!

Instead, the next big thing looks to be autonomous vehicles owned and operated by fleets, who see a strong business case for robocars. This year made it crystal clear that companies such as Waymo, Uber, GM, Ford and others are concentrating on fleet services as a key market for their AVs. GM-owned Cruise app on a smartphone (Photo: Cruise) Phil Magney, founder and principal advisor for Vision Systems Intelligence (VSI Labs), told us, “We’ve seen incremental progress for ADAS and additional sensor technologies for Level 2 cars this year. But as for L4 cars? They are moving much faster” than anyone in the industry expected a year ago. Egil Juliussen, director of research for infotainment and advanced driver assistance systems (ADAS) for automotive at IHS Markit, agreed. Particularly, “Waymo has been quietly taking the lead [in the driverless market],” he said. “They are ahead of everyone else on the field.” Waymo has enough confidence to start putting fully self-driving cars on public roads in Phoenix “without a safety driver.” Juliussen sees this as one of the biggest milestone of 2017. In the following pages, EE Times sums up what we’ve learned in 2017 — both good and bad — in state-of-the-art robocar development. Table of contents: No safety drivers behind the wheel

Robocars vs. human drivers

Intel vs. Nvidia battle heats up

Baidu’s Apollo could change AV landscape

Robocar needs ‘Safety Model’

Waymo’s simulated miles

Geo-fenced, weather-limited

1. Robocars with no safety driver are now on public roads In 2017, Waymo began testing self-driving vehicles on public roads with nobody at the wheel. Although automotive experts were aware of Waymo’s AV advances, many appeared caught by surprise when CEO John Krafcik disclosed last month that Waymo has been operating, since mid-October, its autonomous minivans on roads in Arizona with no safety driver — or any human at all — inside. The headless horseman ride again. Waymo is boldly planning an early rider program, inviting people for free rides in these fully self-driving vehicles. Phoenix-area residents will be Waymo’s guinea pigs. Not coincidentally, Arizona is the rare state where laws regulating autonomous tests are practically non-existent. Waymo’s latest move highlights its growing confidence in its software and hardware advancements. It also signals Waymo’s serious interest in launching a paid fleet service to compete with ride-sharing companies such as Uber and Lyft.

2. Robocars vs. human drivers A self-driven Volvo SUV owned and operated by Uber Technologies Inc. is flipped on its side after a collision in Tempe, Arizona. (Photo: ABC 15) The biggest lesson of 2017 is that “demonstrating safety is the greatest challenge facing the robocar industry,” Demler told us. “It’s not sufficient to estimate how many accidents technology can prevent.” In 2017, we saw a few crashes involving robocars on public roads. Nobody got killed, many were fender benders. In most cases, authorities determined that human drivers, not the robocars, were at fault. However, despite a well-publicized narrative absolving robocars of fault, an Uber crash in Tempe, Arizona, in March and Navya’s autonomous shuttle accident in Las Vegas last month afforded opportunities for experts to examine “actions or inactions” by the AVs involved. Here’s a brief recap of the Uber case in Tempe. Until the police report became available, the common wisdom was that the crash resulted when the driver of a second vehicle “failed to yield” to the Uber car turning left. The initial conclusion was that the accident was caused by a reckless “human” driver. Uber — in self-driving mode — was innocent. However, a police report and eyewitness accounts that became available later showed that the driver who hit Uber’s Volvo was in the intersection waiting to turn left and moving slowly. There was nothing either reckless or sudden. Uber’s Volvo, in self-driving mode, was moving at 38 mph in a 40 mph zone. It failed to detect the left-turning vehicle. Although the Uber’s driver remembers the traffic light changing to yellow as his car entered the intersection, the Uber Volvo didn’t react, neither hurrying nor hesitating. Demler describes the Uber incident in Arizona “an everyday situation, not [even] a corner case.” He said, “The autonomous vehicle technically followed the rules, when it should have been acting responsibly based on the situation that unfolded in real time.” In sum, the “defensive driving” skill common to all good human drivers was conspicuously absent from the Uber Volvo’s programming. Similarly, the autonomous shuttle crash in Las Vegas demonstrated a classic case of robocar inaction. Here, a delivery truck slowly backed up. An automated shuttle, right behind it, patiently waited to get hit by the truck. Again, Las Vegas police quickly declared the accident the truck driver’s fault. The autonomous shuttle was cleared. But afterward, experts wondered why the self-driving shuttle just froze like a deer in the taillights. Why didn’t it lean on the horn to warn the truck driver? Carlos Holguin, CEO of AutoKab, also weighed in. After Navya’s shuttle accident, Holguin concluded “The fault of the truck (human) driver is relative, as we think everything that could be done to prevent such an accident was likely not done, and the shuttle’s system designers were also (at least partly) at fault.” Right now, it seems premature to hold all automated vehicles blameless simply because they know all the rules in the book. Demler suggestd, “We need to develop standards for driving behavior that consider everyday situations, for which rules may not apply (or even exist) but which human drivers can readily evaluate and navigate safely (for the most part) using common sense.” But then, he asked, “How do you program a robot to have common sense?” 2017 has provided us no answer for that.

3. Intel vs. Nvidia battle heats up Intel's CEO Brian Krzanich presented during his keynote at an auto show in LA that the Mobileye EyeQ5 SoC delivers 2.4 TOPS per watt for 2.4 times greater deep learning performance efficiency than Nvidia's Xavier. (Photo: Intel) In 2017, the industry saw a war of words (and specsmanchip) escalate between two dominant AV platform vendors: Intel (Mobileye) and Nvidia. The two giants argued over processing power efficiency in their respective AI chips for autonomous vehicles. As a rebut to Nvidia, which earlier compared its Xavier SoC to Intel’s PC chip, Intel CEO Brian Krzanich said at an auto show in Los Angeles last month that EyeQ 5 — designed by Intel subsidiary Mobileye — “can deliver more than twice the deep-learning performance efficiency” than Nvidia’s Xavier SoC. At CES 2018, Intel is scheduled to unveil an AV platform that combines “the EyeQ 5 SoC, Intel’s low-power Atom SoCs, and other hardware including I/O and Ethernet connectivity” on their platform. In all fairness, though, lacking any details of Intel’s forthcoming announcement, it’s tough to make a meaningful Intel vs. Nvidia comparison on a platform level. While he isn’t sure “how that battle will shape up,” IHS Markit's Juliussen sees certain advantages in Nvidia’s DrivePX platform. Noting the deeper inroads Nvidia has already made among AI-driven AV developers with its DrivePX platform, Juliussen said, “Once you put substantial investment into one platform — especially in building knowledge within software programmers and system designers, you’d find it harder to switch to another platform, unless there is a good reason to do so.” Meanwhile, Nvidia has taken the lead in its own rapidly growing ecosystem, built around DrivePX. Most notable in 2017 was the announcement by Toyota that it will use the Nvidia DrivePX AI car computer platform to power its advanced autonomous driving systems. Asked about the ecosystem, an Intel spokeswoman told us that they, too, have picked up significant partners but can’t publicly disclose who they are.

4. Baidu touts Apollo platform – ‘Android for Robocar’ Tangled web of partnerships in AV industry (Source: EE Times) In the Western world, we have Google. In China, they’ve got Baidu. China came into sharp focus in the AV platform battle in 2017, fueling speculation that Baidu’s influence on the robocar world (and AI) can only grow. Baidu staked its claim in the robocar market this summer by unveiling an open-source autonomous driving program called Apollo. Thus far, 73 companies have joined Apollo. They include not only China’s leading automotive companies such as Chery, Changan and Great Wall Motors, but also many U.S. tech companies and German automakers, among them Nvidia, Intel, Microsoft, Ford, Delphi, Continental, Bosch, Daimler, Velodyne and TomTom. Just this week, NXP also said it’s joining Baidu's Apollo platform. It will offer semiconductor products including millimeter wave radar, V2X, security, connectivity and in-vehicle experience technologies. Notably, the Apollo program is not just for developing China’s domestic AV industry. Companies like TomTom and Microsoft have joined Apollo to offer services outside China. Another aspect of the Apollo project is a $1.5 billion fund — created jointly by Baidu and the Yangtze River Industry Fund — to invest over the next three years in more than 100 autonomous driving projects. As IHS Markit’s Juliussen put it, Baidu is cribbing Apollo straight out of the Android playbook. Baidu’s Apollo is designed to provide everything from cloud service and an open software platform to localized sensor fusion and a hardware reference platform for automated vehicles, he noted. With Apollo’s hardware reference platform, community members presumably can start developing their own applications. Because its reference platform is currently based on Nvidia’s DrivePX, “You could say Nvidia has an inside track,” observed Juliussen. “But that’s not to say that it can be ported to another platform such as that of Intel, because Intel is also an Apollo member.” Baidu gets ready with open-source AV platform (Source: Baidu) In 2017, the AV industry has proven to be a tangled web of partnerships — mostly divvied up among Nvidia, Intel and Waymo, with some relationships firmer than others. However, Baidu with their open-source AV platform approach, could potentially change this landscape. At CES 2018, Baidu is rolling out Apollo 2.0, which is said to enable robocars to do simple urban-road driving.

5. Intel/Mobileye unveils ‘safety model’ for robocars (Source: Mobileye's technical paper “On a Formal Model of Safe and Scalable Self-driving Cars”) Robocars got a shot in the arm when Mobileye published in October a technical paper entitled “On a Formal Model of Safe and Scalable Self-Driving Cars.” This was the first serious attempt by any robocar technology player to establish a mathematical model that absolves an autonomous vehicle (AV) from blame for an accident, “as long as it follows a pre-determined set of clear rules for fault in advance.” The proposal, hailed as a key step to define safety boundaries for robocars, has met some resistance. The Linley Group’s Demler called the Mobileye (Intel) strategy for assigning blame “misguided.” He told us, “You can’t earn public trust by saying ‘Hey we’re not to blame, and we can show you why.’” Academics reached by EE Times nonetheless initially gave Mobileye a positive response. They applauded the company for sticking its neck out and tackling head-on the hardest issue in the robocar debate. Phil Koopman, professor at Carnegie Mellon University, told us, “Overall, I think it's great to see an initial rigorous approach that talks about autonomous vehicle safety. Every vehicle must have some approach to deciding what it's allowed to do and what it's not.” Missy Cummings, a Duke professor who also serves as director of the school’s Humans and Autonomy Lab, agreed. “I appreciate that Mobileye is thinking so deeply about these issues.” But both Koopman and Cummings regard Mobileye’s proposal as only a “first step.” Mobileye’s definition of what might be safe for autonomous cars still needs to be subjected to the rigors of the real world, they said. The advantage of “formal methods and mathematical proofs” is that they can in principle be proven correct. The disadvantage is that “they always have underlying assumptions, and the assumptions might not hold true in the real world,” Koopman noted.

6. Waymo's simulated miles

(Source: Waymo Safety Report)

A safety report — the first of its kind — published by Waymo in October has become a virtual roadmap for automated vehicle developers. Some details in the report show how far the industry still has to go before catching up with Waymo. In its report, Waymo talks about how it designed self-driving software and hardware, and how it tests vehicles. Reading through the report, Juliussen noted, what separates Waymo’s approach from competitors’ robocars is “Waymo is designing their own sensor systems from the software point of view.” After eight years developing driving software, Waymo has learned to “see what’s around the car far better than others,” said Juliussen. Waymo’s ability to tightly couple software mimics the Apple approach, he said. This is something traditional carmakers, who lack software prowess of their own, have difficulty replicating, he added. More significantly, Waymo distinguishes itself through its testing methodology. The company’s safety report declares that Waymo racked up 8.5 million “real world miles on public roads.” To Juliussen, this isn’t the big deal. More important, he said, is simulated miles. Waymo said in its safety report, “In simulation, we rigorously test any changes or updates to our software before they’re deployed in our fleet. We identify the most challenging situations our vehicles have encountered on public roads, and turn them into virtual scenarios for our self-driving software to practice in simulation.” Waymo clocked 2.5 billion simulated self-driving miles in 2016. It increased daily simulation miles from 8 million miles a day in 2016 to 10 million in 2017. “That’s impressive,” said Juliussen, “because in simulations, they are focused on testing only hard cases.” How Simulation Works — Step 1, Step 2 (Source: Waymo) How Simulation Works — Step 3, Step 4 (Source: Waymo) The Linley Group’s Demler agreed. He said, “It’s not realistic to think you can teach a machine common sense by driving more miles. You need massive simulation of test cases drawn from the real world, and along with it the development of autonomous-vehicle standards based on common sense safety criteria.” He said, “The industry is a long way from achieving the latter.” Asked about the overall state of autonomous vehicle development, Juliussen ranked Waymo as the industry leader — by a wide margin. Second is General Motors (who bought Cruise in March 2016), with Ford (which acquired Argo in Feb. 2017) a distant third.

7. Level 4 robocars only go so far: geo-fenced, weather-limited Operational Design Domain: Where Waymo's robocars can go are geo-fenced (Source: Waymo) It’s important to note that Waymo’s robocars today are labeled Level 4, not Level 5. According to SAE’s definition of Level 4 cars, Level 4 is designed for “mind-off” driving. A Level 4 car requires no driver attention for safety. The driver may safely go to sleep or leave the driver's seat. However, self-driving is supported only in limited areas (geofenced) or under special circumstances, like traffic jams. “Outside of these areas or circumstances, the vehicle must be able to safely abort the trip, i.e. park the car, if the driver does not retake control,” according to SAE. As Waymo’s Safety Report makes it clear, Waymo’s robocars adhere to an “operational design domain.” This includes geographies, roadway types, speed range, weather, time of day, and state and local traffic laws and regulations. Waymo acknowledges that an operational design domain “can be very limited.” A typical instance is a single fixed route on low-speed public streets or private grounds (such as a business park) in temperate weather during daylight hours. Waymo, however, is aiming for a broad operational design domain to cover everyday driving. One reason, which perhaps explains Waymo’s keen interest in the ride-hailing business, Juliussen suspects, is that it would provide them with an opportunity for continuous learning and flexible options. As Waymo explained in their report, “Passengers cannot select a destination outside of our approved geography, and our software will not create a route that travels outside of a ‘geo-fenced’ area, which has been mapped in detail.” However, if Waymo enters the riding-hailing business, a passenger requesting a ride outside the approved geo-fence, would simply get a Waymo cab with a driver rather than a robocar. Meanwhile, Waymo can continue mapping new areas and expanding the “fence.” Waymo is eager to tackle adverse weather conditions like different forms of snow. Waymo, which has been reportedly running cold weather testing since 2012, announced that its robocars hit Michigan roads this winter. “This will give our vehicles more practice driving in snow, sleet and ice,” according to Waymo CEO Jon Krafcik. — Junko Yoshida, Chief International Correspondent, EE Times