Technology has never limited its effects to those its creators intended: It disrupts, reshapes, and backfires. And even as innovation's unintended consequences have accelerated in the 21st century, tech firms have often relegated the thinking about its second-order effects to the occasional embarrassing congressional hearing, scrambling to prevent unexpected abuses only after the harm is done. One Silicon Valley watchdog and former federal regulator argues that's officially no longer good enough.

At the USENIX Enigma security conference in Burlingame, California, on Monday, former Federal Trade Commission chief technologist Ashkan Soltani plans to give a talk centered on an overdue reckoning for move-fast-and-break-things tech firms. He says it's time for Silicon Valley to take the potential for unintended, malicious use of its products as seriously as it takes their security. From Russian disinformation on Facebook, Twitter, and Instagram to YouTube extremism to drones grounding air traffic, Soltani argues, tech companies need to think not just about protecting their own users but about what he calls abusability: the possibility that users could exploit their tech to harm others, or the world.

"There are hundreds of examples of people finding ways to use technology to harm themselves or other people, and the response from so many tech CEOs has been, 'We didn't expect our technology to be used this way,'" Soltani said in an interview ahead of his Enigma talk. "We need to try to think about the ways things can go wrong. Not just in ways that harm us as a company, but in ways that harm those using our platforms, and other groups, and society."

Courtesy of Ashkan Soltani

There's precedent for changing the paradigm around abusability testing. Many software firms didn't invest heavily in security until the 2000s, when—led, Soltani notes, by Microsoft—they began taking the threat of hackers seriously. They started hiring security engineers and hackers of their own and elevated audits for hackable vulnerabilities in code to a core part of the software development process. Today, most serious tech firms not only try to break their code's security internally, they also bring in external red teams to attempt to hack it and even offer "bug bounty" rewards to anyone who warns them of a previously unknown security flaw.

"Security guys were once considered a cost center that got in the way of innovation," Soltani says, remembering his own pre-FTC experience as a security administrator working for Fortune 500 companies. "Fast forward 15 or 20 years, and we're in the C-suite now."

But when it comes to abusability, tech firms are only starting to make that shift. Yes, big tech companies like Facebook, Twitter, and Google have large counter-abuse teams. But those teams are often reactive, relying largely on users to report bad behavior. Most firms still don't put serious resources toward the problem, Soltani says, and even fewer bring in external consultants to assess their abusability. An outside perspective, Soltani argues, is critical to thinking through the possibilities for unintended uses and consequences that new technologies create.

Facebook's role as a disinformation megaphone in the 2016 election, he notes, demonstrates how it's possible to have a large team dedicated to stopping abuses and still remain blind to devastating ones. "Historically, abuse teams were focused on abuse on the platform itself," Soltani says. "Now we’re talking about abuse to society and the culture at large, abuse to democracy. I would argue that Facebook and Google didn’t start out with their abuse teams thinking about how their platforms can abuse democracy, and that’s a new thing in the last two years. I want to formalize that."