Employees of tech companies should have the right to know when they are working on projects they may find ethically unacceptable, a former Google whistleblower has said.

In 2018, Jack Poulson hit headlines after he resigned from his job at Google over the company’s (now-scrapped) plan to build a censorship AI for the Chinese search market. Now, he wants to make sure that other tech workers can fight for what’s right without having to put their livelihood on the line.

Poulson has started Tech Inquiry, a non-profit that aims to make it easier for coders with a conscience to speak out inside their companies when they feel ethical boundaries are being crossed. Just as importantly, he’s pushing for greater transparency to prevent workers simply being tricked into doing work that they would never take on voluntarily.

“I believe tech workers need informed consent about when their work may lead to loss of life or suppression of human rights or freedoms,” Poulson told the Guardian in London, where he is due to speak at the Open Rights Group’s annual conference on data and democracy on Saturday. “How is it that we help tech workers who saw something go wrong? How is it that we ensure they have a trusted avenue, somewhere to reach out, that isn’t necessarily going straight to a journalist?”

The goal of Tech Inquiry is to harness the growing wave of employee discontent that has spread across Silicon Valley, and provide workers with the tools they need to stand up to their bosses and the information they need to know when that pushback is necessary. Poulson’s own stand against Project Dragonfly, Google’s effort to build a censored search engine to re-enter the Chinese market, is just one example of that movement. He also cites pushback within Google against Project Maven, a Pentagon AI project that that the company signed up to in April 2017, as well as movements at other companies including Microsoft, Amazon and Intel.

Tech has no moral code. It is everyone’s job now to fight for one | Lizzie O’Shea Read more

Key to Poulson is extending that power to workers in the design and development phases of the industry, where work is often carried out in the greatest secrecy and the public has the least ability to kick up a fuss if it moves in a dangerous or alarming direction. “Internally, and especially in a research department like I was in, there is a feeling that you have a free pass until you’re actually getting ready to hit the button to launch it.”

When Dragonfly was revealed to the public, for instance, “the response from [Google’s chief executive] Sundar Pichai was to try and argue that there should be no accountability because this was ‘just an exploratory project’… And this was not just specific to Dragonfly. I ran into this in YouTube, on a project doing conversational recommendation engines.”

Adults tend not to search conversationally, Poulson explains, instead typing in more structured queries: rather than searching “how do I tie a bow tie”, they may just search “bow tie how to”. Because of that, YouTube had to find a better dataset to focus on more natural queries, and settled on child users.

“One of the project leads is explaining that we’re going to use a data set from YouTube, and from a younger audience, because they use more natural language. I spoke out, and said ‘we can’t use children’s data’, and the response was ‘this is fine, their parents gave the OK’. I spoke to coworkers and they said ‘don’t worry, by the time the thing launches, we’ll have had a thorough privacy review’. When you do R&D, there’s this idea that you can cut corners and have the privacy team fix it later.”

Allowing free ethical rein for “experiments” is dangerous, Poulson argues. For one thing, by the time a project is nearing completion, “you’ve created an entire team whose entire career investment is to launch this thing. And, further, you’ve normalised the behaviour to a bunch of engineers. There’s something fundamental: The first time you see your company doing something, that’s when you’re going to speak out. After that, a switch flips for you, and you accept it.”