NEW YORK -- Uber said on Monday that it plans to buy up to 24,000 self-driving cars from Volvo beginning in less than two years -- and New York's governor wants to bring driverless cars to the Big Apple even sooner.

Mix millions of cars, pedestrians, bicyclists and delivery trucks and you get Manhattan driving. At its best, it's unpredictable. At its worst, it's chaos.

For today's self-driving cars, that mixture might be too much.

"So this cop that's flagging traffic here, right now -- that would confuse the current generation of the self-driving cars?" CBS News transportation correspondent Kris Van Cleave asked.

"Yes, we're not there yet because the brain is not advanced to be able to understand what that person is doing," said Brad Stertz, Audi's director of government affairs.

CBS News drove the city streets with Stertz, who has been working on autonomous driving technology for over a decade.

"So, with the self-driving car, the future of being able to predict that that guy was going to swing out into my lane?" Van Cleave asked.

"Probably not predict, but you would've seen him start to edge over and would have sort of backed off a little bit," said Stertz.

Stertz thinks it could be 2035 before self-driving cars are common in New York City, but Gov. Andrew Cuomo wants autonomous vehicles to get a Big Apple test drive next year. Mayor Bill de Blasio has said "not so fast."

"I really don't like it. I don't think its a mistake. I think it creates a danger," de Blasio said at a news conference.

Self-driving cars use sensors to detect their surroundings and are programmed to follow traffic laws. But a dense city environment might overwhelm the current technology.

"If it weren't for humans, I mean, autonomous vehicles would work perfectly," said Sam Schwartz, the former New York City traffic commissioner.

"The two things that autonomous vehicles have not figured out yet are what are pedestrians about to do and what are bicycle riders about to do," Schwartz said.

This is one vision of how self-driving cars could work in Manhattan: dedicated autonomous thoroughfares leading into the city and cutting across town, potentially allowing some roads to go unused and become green space.

John Meyer, from design firm EDG, entered the idea into a contest organized by New York City.

Getting the pedestrians and the drivers out of the way allows it to achieve true efficiency -- efficiency that will require time, upgraded infrastructure, smarter self-driving cars, and patience in a city not known for it.