Facebook already uses facial recognition software to tag individual people in photos. Apple’s new app, Clips, recognizes individuals in the videos you take. Snap’s famous selfie filters work by mapping detailed points on individual users’ faces. (Snap says on its website that its technology doesn’t take the additional step of recognizing the faces it maps.) That’s similar to how software by the Chinese startup Face++ works. Its software maps dozens of points on a person’s face, then stores the data it collects. The idea is to be able to use facial recognition systems for keyless entry to office buildings and apartment complexes, for example. Jie Tang, an associate professor at Tsinghua University, described to MIT Technology Review how he uses his faceprint to pay for meals: “Not only can he pay for things this way, he says, but the staff in some coffee shops are now alerted by a facial recognition system when he walks in,” and they greet him by name.

It’s understandable, then, that as these technologies rapidly advance, they have become fodder for some conspiracy theories—like the unsubstantiated claim that Snap is building a secret facial recognition database with the images of people who use its popular Snapchat app.

But such conspiracies aren’t as outlandish as they’re made out to be. Experts have been warning against facial-recognition systems for decades. The F.B.I.’s latest facial recognition tools give the agency the ability to scan millions of photos of ordinary Americans. “To be clear, this is a database—or a network of databases—comprised primarily of law-abiding Americans,” said Congressman Jason Chaffetz, a Utah Republican, in a House Committee on Oversight and Government Reform hearing on Wednesday. “Eighty percent of the photos in the F.B.I.’s facial recognition network are of non-criminal entries.” The F.B.I. is able to access images from driver’s licenses in at least 18 states, as well as millions of mugshots.

“Most people have no idea that this is happening,” said Alvaro Bedoya, the executive director of the Center on Privacy and Technology at Georgetown Law, in testimony at the hearing. “The latest generation of this technology will allow law enforcement to scan the face of every man, woman, and child walking in front of a street surveillance camera… Do you have the right to walk down the street without the government secretly scanning your face? Is it a good idea to give government so much power with so few limits?”

The accuracy of the agency’s system is also a matter of debate. According to Chaffetz, roughly one in seven searches of the FBI system returned a list of entirely innocent candidates, even though the actual target was in the database. And the agency doesn’t track its own rate of false positives, according to the Government Accountability Office, which underscores one of the most troubling scenarios for how facial recognition technology could create problems. “It would be one thing if facial-recognition technology were perfect, or near-perfect,” Chaffetz said, “but it clearly is not.”