A few weeks ago, controversy erupted over smart TVs after a series of reports, including one from Consumer Reports, revealed that televisions made by Samsung and others could be recording their owners' private conversations. Implanting microphones inside TV remote controls raises the possibility of some pretty serious privacy violations, but in practice, it seems that Samsung’s TVs are simply processing voice commands on remote servers. That's similar to the way Apple’s Siri, Amazon’s Echo, and many other voice-controlled products work. However, the same TVs are collecting and sharing another kind of data on a massive scale—and in this case the privacy intrusion is very concrete.



You can get a hint of what's going on by closely examining Samsung’s privacy policy.

“By enabling SyncPlus or other marketing features, you may make the content and advertising that you receive on your SmartTV and other devices when you are watching SmartTV more interactive.

“ . . . To make these kinds of enhancements available, we provide video or audio snippets of the program you’re watching to third-party providers that use this information in order to return content or advertising “synched” to what you’re watching. These providers may receive information about your device (e.g., its IP address and device identifiers) and your interactions with the content and advertising they provide.”

What that means is that the TVs are sending data to third parties on everything you watch, whether it's a TV broadcast, a streaming movie, a YouTube video, or a DVD from your private collection. And it’s not just Samsung—LG and Vizio are also harvesting data about their customers’ viewing habits. All these manufacturers have been embedding the functionality into many of their smart TVs, in some cases since 2012.

The technology is known in the television business as automatic content recognition, or ACR, and it has spawned an entire secondary industry focused on collecting and monetizing information about the viewing behavior of TV viewers directly from their own television sets.

Here’s how it works: Companies such as Cognitive Networks, Enswers, and Gracenote collaborate with television manufacturers to embed ACR technology into smart TVs that monitors either the video or audio stream—and sometimes both—that the user is watching. The ACR creates a “fingerprint” of the on-screen content, then sends it to a remote server that uses that fingerprint to determine what programming is being watched.



Since much of the ACR process is handled by these third parties, it is likely that millions of smart TV owners have inadvertently left an extensive data trail chronicling months, if not years, of their TV-watching history on the servers of companies they’ve never heard of.

Precise data about consumer TV viewing habits can be very valuable information. Revenues for audience-measurement company Nielsen surpassed $6 billon last year. Television manufacturers have promoted ACR as an interactive-TV technology—LG, for instance, partnered with cable channel Showtime to synchronize on-screen extra content during shows such as “Homeland” and “Shameless." But the companies behind ACR also see its potential as an analytics and targeted-advertising platform.

Enswers has been embedded at the hardware level into Samsung smart TVs since 2012. The company has already used the technology to push interactive advertisements for retirement-planning financial products in Spain, and to prompt Samsung smart TV owners to purchase David Beckham underwear during Super Bowl XLVIII using their remote controls—a use of ACR that has come to be known as “t-commerce.”