Consumer Reports tested Tesla’s recently launched Smart Summon feature on a Model 3, and found that the automation was glitchy and at times worked intermittently, without a lot of obvious benefits for consumers.

CR tested the feature over several days at its Auto Test Center in Colchester, Conn., and in nearby parking lots. The analysis follows a flurry of social media posts from Tesla owners critical of the feature.

Tesla says Smart Summon is one of the first products in a suite of technologies it markets as “Full Self-Driving.” The automaker activated the feature for car owners using an over-the-air update at the end of September.

In online marketing materials (and in public comments from CEO Elon Musk), Tesla says owners using a smartphone can summon their vehicles to come pick them up to help in everyday situations, such as to avoid walking across a parking lot in the rain or with an armful of groceries.

The Model 3 owner’s manual contains numerous warnings of Smart Summon’s limitations, including that it can’t be used on public roads and can’t detect all traffic or curbs. But some owners on social media have reported minor fender benders while using the feature in parking lots and at low speeds, CR noted.

Smart Summon has drawn the attention of the National Highway Traffic Safety Administration, which said it’s in contact with Tesla to gather information about the feature.

Jake Fisher, CR’s senior director of auto testing, says consumers are not getting fully tested, consumer-ready technology. In essence, he says, Tesla owners are being enlisted as beta testers to help fine-tune the technology for the future—even though they’re paying $6,000 up front for the promised automation.

What consumers are really getting is the chance to participate in a kind of science experiment. This is a work in progress. —Jake Fisher

For more than a year before its release, Musk stoked interest in Smart Summon with public comments, describing how Tesla owners would be able to use their vehicles like remote-controlled toys that could follow car owners around like pets.

CR’s experience with the system shows that Smart Summon can exit a parking space, turn and start moving toward the vehicle owner, and negotiate around stationary objects. It also can detect and stop for pedestrians and slow down if it senses cross traffic.

But CR also found that the system works only intermittently, depending on the car’s reading of the surroundings. The system is designed to work only in private parking lots, but sometimes it seemed confused about where it was. In one case, the system worked in one section of a private lot, but in another part of the lot it mistakenly detected that it was on a public road and shut itself down. At various times, the Model 3 would suddenly stop for no obvious reason.

When it did work, the Model 3 appeared to move cautiously, which could be a positive from a safety perspective. But it also meant the vehicle took a long time to reach its driver. The Model 3 also didn’t always stay on its side of the lane in the parking lots.

Over several days, CR repeatedly asked Tesla for comment. CR has called the company and also emailed multiple written questions asking about the Smart Summon technology. The company hasn’t yet returned the calls or responded to the questions.

Separately, CR added, NHTSA published a petition late last week from a group of Tesla owners about an alleged battery defect in the Model S and Model X. According to the petition, Tesla released a software update last year that reduced the possibility of battery fires but failed to disclose the alleged defect to NHTSA as required by law. NHTSA says it is deciding whether a formal defect investigation is needed.