The first time Neal Levy gave up control of his car, it felt surreal.

He’s the proud owner of a 2012 Tesla Model S Signature Edition, one of the very first to ship to U.S. customers. These models came out before the driver-assisting Autopilot technology debuted. But when he brought it in for repairs, he received an Autopilot-enabled loaner.

“I got onto the street where my office is and turned on Autopilot. It was a 35 mile-an-hour road, that bent pretty sharply to the left, and I kind of just held on,” he tells Inverse. “And, you know, I had my hands on the steering wheel and my foot on the brake pedal in order to — just in case — take over.”

The car didn’t crash, but Levy’s caution is very much the norm. A Kelley Blue Book poll in 2016 found that 51 percent of respondents would rather have full control of their vehicle, even if it would not be as safe as an autonomous one. And a AAA survey from the same year found a staggering 75 percent of respondents were afraid to ride in an autonomous car.

Not everyone has the same experience as Levy, though. Tesla owner Chris Hightower, who lives in Dallas, owns a 2013 Tesla Model S without Autopilot, but over the holidays last year, got to try one with the latest technology.

“The round trip to my parents’ house was around 500 miles, and 50 of that was off highways and onto farm roads,” Hightower tells Inverse. “I used it extensively, without any issues, even through stop and go traffic.”

Experts say it’s best to think of self-driving cars as a patchwork of technologies, and developing trust in each one will be vital to putting your faith in the system and cooly taking your hands off the steering wheel of a semi-autonomous vehicle. For example, an autonomous car needs to brake in an emergency. If you can trust the car with that task, you can cross it off the list. Automatic emergency braking (AEB), already in cars now, applies the brakes automatically if a human fails to do so. One example of AEB is Ford’s pedestrian detection system which fuses video with radar and feeds that information into an algorithm that lets the car know if a collision is imminent.

Tesla’s Autopilot is a more evolved version of AEB technology, but the principles are still the same. As it takes care of highway cruising, it represents the first baby steps toward a fully autonomous system, where drivers would give up control entirely. A future update to Teslas powered by “Hardware 2” sensors, intended for launch by the end of this year, promises full autonomy in certain situations.

“Autonomous driving is not going to be here overnight,” Jeremy Carlson, senior analyst with the automotive division of IHS, a London-based financial analytics firm, tells Inverse. “It’s going to be a gradual rollout of technology, and it’s going to take some time from even today before we have fully autonomous vehicles.”

“It was easy to learn the limitations of the sensors after maybe ten minutes of playing with it,” Hightower says of his self-driving Tesla road trip. “I am a big fan, and truly believe Tesla has cornered the market with the Hardware 2 gear that started shipping this past November.”

Meanwhile Levy thinks he’ll one day feel ready to give up control. He just needs to get used to it.

“I still want to witness it myself for a while, and know the car more intimately than ‘Here’s your keys,’ and I just set a destination and let it go,” Levy says. “I would probably first do a bit more driving on my own and progressively let it get there before I more blindly allow it to drive me.”

The inside of a Tesla. Getty Images / Spencer Platt

These results are not surprising. In the Kelley Blue Book poll, respondents that tried limited autonomy reported warmer feelings toward driverless car tech than those that hadn’t. Similarly, in the AAA survey, drivers who owned semi-autonomous vehicles were 75 percent more likely to trust driverless cars than those that didn’t.

Even Levy, who was slightly more cautious than Hightower, believes in autonomous tech. “I have a two-and-a-half-year-old daughter,” he says. “I’m assuming at this point that she will not drive a car.”

Analytics firm IHS Automotive predicts that in the year 2035, the industry will sell 21 million autonomous cars. Before then, most of the growth from autonomous cars is expected in ride-sharing or mobility on demand. The firm is projecting 76 million semi-autonomous cars will be sold between now and 2035, which will help sell consumers on the idea. By that time, Levy and others will probably feel less like they have to hold on for dear life, as the technologies will have grown more commonplace.