CO

It’s funny, I don’t actually think of Weapons of Math Destruction as a book about math. It’s more a book about instruments of social control, and how they are masquerading as mathematical.

I wrote the book because I think there is considerable harm being done by destructive algorithms, and as a mathematician I’m in a unique position to explain those harms. I worked as a hedge fund quant during the 2008 financial crisis and as a data scientist at the height of the big data revolution. So I have been living behind the scenes, and I know how this stuff works.

At the same time, I’m an occupier. I joined Occupy in October 2011, forming and facilitating the alternative banking group, which has met weekly at Columbia University since then. Our weekly discussions have established a lens through which I’ve learned to examine the world, especially as it connects to money and power.

So now when I come across an automated decision-making system, I always wonder who is benefiting from that system, and who is suffering. And the conclusion I keep coming to when considering systems that rely on algorithms, is that poor people, black and brown people, and the mentally ill are consistently being shut out by these algorithmic black-box structures.

Machine learning has been presented to us as trustworthy, because it’s mathematically sophisticated and because algorithms have no agendas. But the data itself cannot be decontextualized from our historical practices, nor can the choices of the modelers who build the models and choose objective functions.

In other words, we don’t move past discrimination through the use of algorithms, but rather instead sanitize and obscure our historical-cultural practices and patterns. In the process we risk reinforcing and even exacerbating those patterns.

I wanted to share this truth with people, so I quit my data science job in the summer of 2012 and devoted myself to writing a book about destructive algorithms and how they shape our society.