This is such an extreme outlier I'm ready to believe that a nation state actor may have an expert AI team* reverse engineering and gaming social graphs and recommendation algorithms at scale silently.

* Asymmetric black hat research (see below) and/or StuxNet level competent

1/N

It's quite possible for machine learning to have exploits as fundamentally severe and retrospectively obvious as the NSA's 13+ year head start in differential cryptography. White hat research is a terrible proxy for black hat research - especially for AI.



Our social media companies have issues:

- Most social media companies aren't yet competent at blocking even obvious issues let alone adversarial

- There's little to no transparency for good actors (researchers, journalists, ...)

- Profit motives above all



Even with minimal access good actors have found many fundamental flaws in these ecosystems. Flaws that _should_ have been painfully obvious. These good actors generally have nearly no resources. How competent might a black hat researcher be in their place?



White hat researchers - and even grey hat researchers - are fundamentally not the same attack model as black hat researchers. How many of these companies have machine learning red teams? How many are given permission to scrape from a million botnet'ed IPs?

https://en.wikipedia.org/wiki/Red_team#Overview …

How many red teams spawn a million bot accounts? How many manipulate real people and coordinate with public figures? How many are allowed to slander or continuously attack individuals in a given clique?

---

The answer is likely zero for both practical and ethical reasons.

Black hat researchers have none of these limitations. Low hanging but ethically dubious fruit - from data breaches to manipulating real people - are all ripe for them to pluck. They can play with variables white/grey hat researchers could only dream about in ethics reviews.

In machine learning simple techniques can be scarily effective when you have the right ingredients. How effective could we be if we dropped our ethical foundation? We'll never answer that ourselves but our black hat research cousins may be publishing arXiv papers in plain sight.

You can follow @Smerity.

Share this thread

Bookmark

____

Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.



Enjoy Threader? Sign up.



Since you’re here...



... we’re asking visitors like you to make a contribution to support this independent project. In these uncertain times, access to information is vital. Threader gets 1,000,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Your financial support will help two developers to keep working on this app. Everyone’s contribution, big or small, is so valuable. Support Threader by becoming premium or by donating on PayPal. Thank you.



Download Threader on iOS.