There were even passes that violated state laws, officials said. It would pass on the right on a two-lane divided highway, or wouldn't move back to the right-side travel lane after a pass.

A Tesla spokesperson stressed that it was the "driver's responsibility" to maintain control, and that extended to "safely executing lane changes." That's consistent with past policies. The company has long included mechanisms to ensure you're paying attention. Automatic lane changes, meanwhile, require your consent before they're turned on.

However, that wasn't enough for CR. The outlet argued that companies should need to provide public evidence of a semi-autonomous system's safety before it's made available to buyers. It also contended that Navigate on Autopilot is more trouble than it's worth -- it's akin to "monitoring a kid behind the wheel for the very first time," CR's Jake Fisher said. While Tesla was still ahead of rivals on autonomy, CR thought that it was safer and more relaxing to perform lane changes yourself.

Update 5/24 11:30AM ET: We've updated the post to make it clearer that automatic lane changes aren't enabled by default.