The problem, according to Bloomberg's sources, is the lack of transparency. Contractors from TaskUs reportedly weren't told where the audio came from or why they were transcribing it. That led some of the workers to believe their work was "unethical," especially when some of the conversations included vulgar material. Facebook's data privacy policy also doesn't make clear that human beings might monitor content.

It's not surprising that Facebook would have frozen transcriptions, at any rate. As with voice assistants, there's a fear that staff might listen to sensitive information and potentially abuse that for their own ends. The company has long had to contend with allegations of eavesdropping on phones to target ads -- Messenger transcriptions weren't going to help with public perception. And when Facebook only just received a $5 billion FTC fine for privacy violations, it likely didn't want to risk further government scrutiny.

The challenge is to balance privacy with technical needs. AI transcription still has a lot of room for improvement, and it's difficult to boost accuracy without good examples. If Facebook has to permanently end its transcription program, it may have to either scramble for a viable alternative to improving the AI or else accept that its accuracy might remain similar for a long time.