Self-driving cars have one thing in common with human drivers: They can’t spot cyclists. While car recognition has improved a lot in recent years, the detection of the most vulnerable road users is relatively poor. That’s partly down to a lack of training, and partly down to the bikes themselves.

Cars are easy to spot. They’re more or less uniform boxes, and have clear design indicators that tell you which end is the front, making it possible to recognize a car and its direction of travel even in a 2D image. Bikes, on the other hand, vary almost as much as their riders. They’re also small, they weave unpredictably, and they change shape drastically when you add luggage. And even when a car does recognize a bike, it often can’t tell which way it is pointing.

But don’t worry, people are hard at work on the problem. Deep3DBox, for instance, is an algorithm from researchers at George Mason University and robot taxi company Zoox. Deep3DBox looks at a 2D image and finds the vehicles therein, surrounding them with a 3D box, and determining what direction the car is moving in. It correctly identifies 89% of cars, writes IEEE Spectrum‘s Peter Fairley, up from the usual 70% rate common a few years ago. But even Deep3DBox has trouble with bikes, spotting just three-quarters of them, and only orienting 59% of them correctly.

There are a few ways to improve this. LIDAR and radar scanning lets a car build up a true 3D picture of the road ahead, and they can also see movement, which helps work out which way things are pointing. Another leg up comes from super-detailed 3D maps, which include everything on and around the road, including street signs and lane markings. Armed with such a detailed backdrop, a car can quickly spot anything that isn’t usually there–something like a bike, for example.

But bikes bring another problem that’s less easy to solve. Or rather, their riders do: unpredictability. A bike is quick and agile, and a rider will often dart into traffic, or lurch across lanes. And no matter how good your detection algorithms, when a bike appears from nowhere, panic might be the only option.

“Bicycles are much less predictable than cars,” Deep3DBox contributing author Jana Košecká told IEEE Spectrum, “because it’s easier for them to make sudden turns or jump out of nowhere.”

Even so, autonomous bike detecting will one day be better than human bike detecting (if it isn’t already), and the technology is already being deployed in regular, human-steered cars. Jaguar already has a bike-warning system in its cars, and Volvo has added bike detection to its pedestrian-sensing technology.