We’re still a long way from faultless driverless cars.

American organisation the Insurance Institute for Highway Safety (IIHS) has put some of the latest semi-autonomous driver aids under the spotlight and most didn’t fare well.

The IIHS took five new vehicles – the Tesla Model 3 and S, BMW 5-Series, Mercedes E-Class and Volvo S90 – and evaluated their performance both on a test track and in real-world situations.

Specific driver safety technologies examined were automatic emergency braking (AEB), adaptive cruise control and lane keeping assist. All three safety aids are becoming increasingly common as either optional or standard equipment on models from most manufacturers.

Beginning with AEB, which is mandatory in Australian vehicles that hope to score a five-star ANCAP rating, all vehicles were driven at a speed of 50km/h towards a stationary object. In a surprise result, not all vehicles passed the test. The Model 3 and S, running Autopilot software versions 8.1 and 7.1 respectively, both struck the object.

Each of the high-tech machines are billed as having SAE Level 2 autonomous ability. Out of the five autonomous levels, Level 2 is temporary hands off/feet off driving but it still requires the driver to always be ready to take over.

Tesla CEO Elon Musk said in a tweet in June this year that he expects Autopilot version 9.0 to be fully driverless: “To date, Autopilot resources have rightly focused entirely on safety,” Musk tweeted. “With V9, we will begin to enable full self-driving features.”

The two Teslas did however do better in all other tests.

With adaptive cruise controlled activated on the same test all vehicles except the Volvo braked early, smoothly and stopped correctly. But on a road following traffic there were issues.

“Out on the road, engineers noted instances in which each vehicle except the Model 3 failed to respond to stopped vehicles ahead,” the IIHS said in its findings.

In one situation, the E-Class failed to recognise a stationary pick-up truck.

“Traveling about 55 mph [88km/h] with ACC and active lane-keeping engaged but not following a lead vehicle, the E-Class system briefly detected a pickup truck stopped at a traffic light ahead but promptly lost sight of it and continued at speed until she [the safety driver] hit the brakes.”

The Volvo S90, while stopping in all circumstance and which touts a similar level 2 Pilot Assist package, failed to brake smoothly – though it never struck the parked object or trailing car. Instead, the IIHS reports that the Swedish vehicle stopped very late with just 1.1 seconds before impact developing up to 1.1g of force.

And the Model 3 itself was not immune to scrutiny, reportedly braking too much: “Unnecessary or overly cautious braking is an issue IIHS noted in the Model 3. In 180 miles [almost 300km], the car unexpectedly slowed down 12 times, seven of which coincided with tree shadows on the road. The others were for oncoming vehicles in another lane or vehicles crossing the road far ahead.”

Beyond ACC and AEB tests, active lane keeping assist was also evaluated. The technology is able to decipher road markings and sometimes curbs to automatically steer the vehicle so that it doesn’t depart the lane. It’s another safety assist technology which is becoming increasingly common in new vehicles.

In this test, the Tesla Model 3 proved most accurate, but a combination of environments tricked all vehicles.

“Only the Model 3 stayed within the lane on all 18 trials. The Model S was similar but overcorrected on one curve, causing it to cross the line on the inside of the curve in one trial. None of the other systems tested provided enough steering input on their own to consistently stay in their lane, often requiring the driver to provide additional steering to successfully navigate the curve.

“The E-Class stayed within the lane in 9 of 17 runs and strayed to the lane marker in five trials. The system disengaged itself in one trial and crossed the line in two. The 5-Series stayed within the lane in 3 of 16 trials and was more likely to disengage than steer outside the lane. The S90 stayed in the lane in 9 of 17 runs and crossed the lane line in eight runs,” the report said.

On-road tests were better for some vehicles, where the E-Class’ lane keeping assist stayed in its lane 15 out of 18 times and the Model 3 stayed in its lane all but one time when it hugged the marking. All other vehicles ‘struggled’ according to the report.

“In contrast, the 5-series, Model S and S90 struggled,” it said.

“The 5-series steered toward or across the lane line regularly, requiring drivers to override the steering support to get it back on track. Sometimes the car disengaged steering assistance on its own. The car failed to stay in the lane on all 14 valid trials.”

“The Model S was errant in the hill tests, staying in the lane in 5 of 18 trials. The S90 stayed in the lane in 9 of 16 trials.”

In conclusion, the IIHS said the evidence for the safety benefits of technology such as AEB and ACC is more compelling than the current level of active lane-keeping assists.

Technology such as AEB and ACC has had more time to develop and mature and the former is a requirement for a five-star ANCAP rating in Australia. The safety authority has also announced it will add performance testing of safety assist technology including electronic stability control, speed assist systems, lane support systems and autonomous emergency braking which could expose poorly configured systems.

The IIHS is also looking to move towards a consumer rating system for advanced driver assistance systems as a segue to driverless technology.

“Apart from questions about whether the systems perform as drivers expect, one of the many factors to consider is how much of the driving task can safely be handed over to technology without drivers checking out altogether?”