Decades ago, Congress mandated that federal authorities keep track of foreign nationals as they enter and leave the United States. If the government could record when every visitor stepped on and off of U.S. soil, so the thinking went, it could easily see whether a foreign national had overstayed a visa.

But in June of last year, without congressional authorization, and without consulting the public, the Department of Homeland Security started scanning the faces of Americans leaving the country, too.

You may have heard about new JetBlue or Delta programs that let passengers board their flights by submitting to a face recognition scan. Few realize, however, that these systems are actually the first phase of DHS’s “Biometric Exit” program.

For certain international flights from Atlanta and New York, DHS has partnered with Delta to bring mandatory face recognition scans to the boarding gate. The Delta system checks a passenger is supposed to be on the plane by comparing her face, captured by a kiosk at the boarding gate, to passenger manifest photos from State Department databases. It also checks passengers’ citizenship or immigration status. Meanwhile, in Boston, DHS has partnered with JetBlue to roll out a voluntary face recognition system for travelers flying to Aruba. In JetBlue’s case, you can actually get your face scanned instead of using a physical ticket.

While these systems differ in details, they have two things in common. First, they are laying the groundwork for a much broader, mandatory deployment of Biometric Exit across the country. Second, they scan the faces of everyone—including American citizens.

Treating U.S. citizens like foreign nationals contradicts years of congressional mandates. DHS has never consulted the American public about whether Americans should be subject to face recognition. That’s because Congress has never given Homeland Security permission to do it in the first place. Congress has passed Biometric Exit bills at least nine times. In each, it has been clear: This is a program meant for foreign nationals. In fact, when President Trump issued an executive order in January on Biometric Exit, it was actually reissued to clarify that it didn’t apply to American citizens.

Behind the scenes, DHS is already handling your face recognition photo in ways many travelers might find alarming. For instance, after JetBlue scans your face, your photo is temporarily stored in DHS’s threat modeling ecosystem. What is it doing there? While there is no indication right now that DHS is, for example, comparing your face against a hotlist of known or suspected terrorists, it’s easy to imagine DHS pulling the trigger. People with foreign-sounding names have already struggled for years with false matches on the No Fly List. Will people with “foreign-looking” faces encounter the same discrimination?

And this may only be the beginning. According to U.S. Customs and Border Protection’s John Wagner, Homeland Security is in internal negotiations to bring face recognition to the TSA security checkpoint.

What might mission creep look like? One possible scenario involves DHS deciding to search your face against state and local law enforcement databases. Would you be comfortable with your face being compared to the faces of wanted criminals simply because you flew home to see your parents? Or maybe DHS could decide to share your face with the FBI. That could mean your face being compared with unknown suspects in security camera footage. Imagine being investigated for a crime you didn’t commit because, while passing through the airport, an algorithm matched your face to a suspect in a grainy surveillance video.