The privacy debate intensifies as brands embrace biometric tech.

Alibaba's FlyZoo hotel uses facial recognition technology in place of access cards and room keys.

Touted by retailers, hoteliers, and even schools as a mechanism for customer service and convenience, facial recognition technology is rapidly expanding beyond surveillance and security to become a powerful optimization tool for brands. Yet, its future teeters on a tidal wave of pushback from consumers and advocacy groups concerned about the ethics of the technology. As the market for facial recognition software alone is predicted to reach $9 billion by 2022, according to Market Research Future, a recent poll by the Pew Research center suggested only one-third of Americans trust technology companies to use facial recognition responsibly.

Facial recognition companies have been licensing out their software for a broad range of applications. While Apple most notably has used it for its “Face ID” system for unlocking the iPhone X and making payments as early as 2017, an AI startup in Texas called Waldo, launched in 2016 to employ the technology at schools and summer camps. Waldo’s facial recognition service collects snapshots of children during events and activities, which they send to the parents as a yearbook alternative.

Retailers in the U.S. have been using facial recognition software to track consumer behavior. One mall in California set up cameras operated by Las Vegas-based Remark Holdings Inc to track shoppers’ movements throughout stores and gather information that would ultimately be used to “make the shopping experience better for customers,” while Remark Holdings’ other retail clients use the data to craft loyalty programs. California-based fashion brand Ruti uses biometric data garnered from cameras in their stores to immediately inform customer service decisions—“The technology automatically pulls five to six items from the most recent collection, with pictures of those items, and shares recommended sizes for the sales associate to pull to help ensure the customer will have an easy try-on experience in the dressing room,” wrote Katie Richards for Glossy.

FlyZoo's mobile check-in using facial recognition

Two Marriott hotels in China use the technology to help Chinese guests conveniently skip the line at the front desk and check into their rooms using their face. Meanwhile, in Hangzhou, FlyZoo, a new “futuristic” hotel from tech giant Alibaba, uses biometric data to replace room key cards entirely.

Elsewhere in China, Alibaba’s ‘Smile to Pay’ service (later upgraded to a smaller, more efficient “Dragonfly”) scans customers’ faces at places like KFC, grocery stores, and bakeries for a more seamless checkout experience sans smartphone. Megvii, the company behind the Smile to Pay, which filed for a listing on Hong Kong’s stock exchange in August, is also responsible for a range of other applications that use its Face++ technology, including Lenovo laptops, a ride hailing app, and security cameras.

Smile to Pay is currently in use across more than 300 cities, but businesses will soon have another option from rival Tencent. The company recently demoed its answer to facial payment in August, “Frog Pro”, designed to be compatible with WeChat Pay, which already works with 50 million merchants in China. Facial recognition is also deployed as part of China’s new smart recycling program—in this case, the technology is used so waste bins only open for faces the system recognizes from the neighborhood. It also helps collect data to reward those who are sorting their trash correctly, and penalize those who aren’t.

Smile to Pay at KFC China

Accountability is, in fact, a common application of the the technology. Schools in Guizhou and Beijing are using facial recognition scanners to make sure their students aren’t playing hooky. In one instance, they’re designed to ensure that students don’t swap their “smart” uniforms designed by Guizhou Guanyu Technology, which contain a chip that will set off an alarm if the wearer leaves the school building and send information to their parents about what time they came to class. In another case, a university professor requires students to check in by scanning their faces to make it easier to keep track of who skipped class.

Beijing’s newest airport is also using facial recognition as a security measure and to assist passengers with finding their gates. And a few weeks ago, Intel announced it would be implementing a “large-scale” facial recognition system to speed up the security check-in process at the 2020 Tokyo Olympics, marking the first time it would be used for the international sporting event.

Biometrics backlash

Baidu facial recognition

Around the world, these wide-reaching applications of facial recognition and societal surveillance have led to an outpouring of criticism. U.S. digital rights advocacy group Fight for the Future is petitioning for a nationwide ban that prevents governments from using facial recognition. Fight for the Future’s deputy director, Evan Greer, said in a statement, “This surveillance technology poses such a profound threat to the future of human society and basic liberty that its dangers far outweigh any potential benefits. We don’t need to regulate it, we need to ban it entirely.”

Cities are already passing their own bans on facial recognition. San Francisco became the first city to halt police use of facial recognition technology in May, and Somerville, Massachusetts and Oakland, California followed suit soon after. California state lawmakers passed legislation that prohibits the use of the technology in police body cameras; in London, the developer of King’s Cross recently abandoned its use of facial recognition in its surveillance cameras following a backlash in response to a statement released in August that it was using the technology, though a British high court ruled in September that facial recognition was lawful for the South Wales police force.

The backlash is even trickling down to use cases outside law enforcement. Only a year after Ticketmaster’s parent company Live Nation Entertainment announced it would use facial recognition scanners to replace ticket machines at its concerts, a wave of musicians, including Rage Against The Machine’s Tom Morello, protested, citing its potential to put at-risk communities in potentially unjust or harmful situations. In fact, numerous studies have revealed the biases and inaccuracies with facial recognition technology, including a 2018 test by the ACLU where Amazon’s facial recognition software misidentified 28 members of Congress as criminals.

In China, concerts have been a resource for police. Last year, police located a 31-year-old wanted for “economic crimes” among an crowd of 60,000 at a Jacky Cheung performance. And though facial recognition technology seems to be a more widely accepted phenomenon across the nation, reaction to a China Mobile ad released last spring for its 5G network suggests a more nuanced state of affairs. The two-minute spot depicts a scene fit for a crime film trailer: a detective wearing facial recognition glasses scans a crowd for a suspect, and detects and pursues him in spite of the suspect donning a rather unusual disguise. “Does this advertisement promote 5G or terror?” a lawyer asked on Weibo, as reported by Radii China. “As technology increasingly intrudes into our personal space, the protection of personal privacy by legal boundaries must keep pace. While attacking a very small number of criminals, we should protect the right to normal life of the overwhelming majority of good people.”

Several companies appear to be hearing and heeding these swelling concerns. Microsoft, one of the leading firms that supply artificial intelligence algorithms with an extensive database of faces, recently deleted it, not long after it was revealed that Megvii, among other facial recognition companies, were using it to test their software.

Facing the future

FlyZoo facial recognition room access

The debate over lawful use of facial recognition technology shows no sign of abating—at the end of September, a number of technology companies and police groups wrote an open letter to Congress urging them to “consider many of the viable alternatives to bans so that law enforcement can use facial recognition technology safely, accurately, and effectively.” The aforementioned poll from the Pew Research Center shows that 56 percent of Americans actually trust law enforcement to use facial recognition “responsibly.” And approximately 80 percent of Gen Z consumers reported expecting biometrics to be part of their digital experience within the next five years, though a recent UK study suggests this group highly values their digital privacy.

And, still, U.S. lawmakers proposed a bill in June that would, if passed, bar Chinese companies from the Face Recognition Vendor Test, a widely accepted method for testing the accuracy of facial recognition software. The draft addresses human rights groups’ concerns over facial recognition’s threat to civil liberties and its role in the surveillance of ethnic minority groups.

Brands that do adopt facial recognition technology may run into challenges other than privacy concerns. Retailers would want to be cautious about putting their customers at risk of fraud or identity theft should their biometric database become subject to hackers. More companies are also opting to be transparent about using the technology and asking consumers for consent—Facebook, for example, recently began giving users the option to turn the facial recognition feature on the app on or off depending on their personal preferences. This came as they reached a $5 billion settlement with the Federal Trade Commission after it accused the company of deceiving users over their privacy settings.

Meanwhile, in response to a slightly more unusual consumer complaint, Alibaba even elected to add beauty filters to their facial recognition service on Alipay. “We are going to make you look even prettier than with a beauty camera,” the payment service announced on its social media page in July. “I bet you’ll be impressed.”