TLDR: Do it if you enjoy it, don’t do it for the money

“A.I. Researchers Are Making More than $1 Million at a Nonprofit” declares the NYT [1]. It’s certainly not the first article where commentators have opined upon how much top AI researchers make and how it reflects the rosy economics of young people going into a career in machine learning or data science.

The article reports that OpenAI co-founder, Ilya Sutskever, was paid USD 1.9mio and Ian Goodfellow, the inventor of GAN, was paid USD 800k in 2016. While the 800k that the AI superstar pulled in is nothing to scoff at, especially for the average thirty-something year-old, hundreds of relatively unknown quants in finance are paid this much every year. In fact, in 2003 I left my first job to join Wall Street because a senior manager showed me a Financial Times advertisement:

For Hire: Hedge Fund Quantitative Analyst

PhD in Mathematics, Statistics, Physics, Econometrics

Mathematics Olympiad winner or similar

£500k base + discretionary bonus

In his heyday, a big name like Emanuel Derman would have been paid much more than 800k (perhaps an order of magnitude more). So, rather than be a cause for bullishness, the numbers suggest to me that AI researchers and data scientists are underpaid relative to their quant brethren.

As a quantitative trader, and an erstwhile algorithmic trading quant, I think I’ve seen this movie before and I believe it should be more accurately interpreted as a continuation of a long trend of high-tech coolies coding themselves out of their jobs upon a backdrop of global oversupply of skilled labour that is now fueling right-wing political movements in developed economies.

The First Quants

The first Wall Street quants worked on derivatives — the most famous of whom were academic luminaries like Fischer Black, Myron Scholes and Robert Merton (of Black-Scholes formula fame) and later on, industry practitioners such as Emanuel Derman. Derman continues to have a long and illustrious Wall Street career while Scholes and Merton became (in)famous, making and losing billions through their involvement in the Long-Term Capital Management saga during the 1998 financial crisis, which not only brought down their fund, but was implicated in almost bringing down the global financial system. Their academic successors, armies of physics, mathematics and economics PhDs drained out academia (a common accusation in the AI and robotics space today) and moved to Wall Street between the late 1990’s to 2007 to create ever more complex derivatives using increasingly sophisticated mathematical and numerical methods.

But within the space of about 15 to 20 years, options quants managed to largely model themselves out of existence (there are only so many double lookback Himalayan options one can invent). The 2008 financial crisis hastened their decline, and desks of 20 or more derivative quants and structurers shrank to teams of one or two.

Not just quants…

The Second Wave of Quants: Algorithmic Trading

As a computer scientist and ANN enthusiast with an interest in market micro-structure, I was part of the second wave of quants to hit Wall Street in the early 2000’s. By this time, PhDs in mathematics or computer science on a trading desk were no longer a rare sight, and at a time when exchanges became fully electronic, tattooed truck drivers hustling the pits were beginning to see the writing on the wall. We turned their fears into reality.

Contrary to the industry outsider’s idea of what “algorithmic trading” is, algorithmic trading doesn’t involve making money by deciding what to buy or sell. Algorithmic trading quants worked mostly in banks and our mission was to automate the manual art of executing large orders by splitting them into small chunks and sending them to the stock exchange in a way that minimised our information leakage and market impact. The better we were at minimising the cost of buying and selling stocks, the more business our hedge fund clients would give us.

As a twenty-something, my boss at the time told me that the algorithms we were creating would replace the sales traders, who were responsible for talking clients into giving us orders and giving them to the dealers (execution traders) to actually work in the market. This assertion always struck me as strange, and it was only later that I realised my manager was only saying that to reduce the suspicions of the dealers whom he also managed. Within a few short years, as the first bulge bracket investment bank to invest heavily in algorithmic trading, our firm went from having a 4% market share to a 10%+ market share in Hong Kong. During this time, the number of execution dealers fell from from six to two.

Trading and execution before algorithms

For a junior quant, those were exciting days, and we felt like we were the new Black and Scholes. While the derivative quants worked on boring variations of 40 year-old ideas, the field of market micro-structure, the study of short-term supply and demand, and market design, was in relative infancy. There were very few people in investment banks with this knowledge and there was no real consensus on how to even measure trading performance. Top academics would reach out to us to obtain data on trades and exchanges consulted us for advice. We were developing new products with outrageous names like Sniper and Guerrilla (yes, really) and soon every other investment bank would follow with similar strategies.

While even today, many problems regarding order book dynamics remain unsolved (at least in the public domain!), many of my ex-colleagues in algorithmic trading now work on staid variations on the theme. We automated the dealers, and eventually ourselves out of existence in the space of about ten years.

Buy-side Quants

The third class of quants, some say the most prestigious group, work on the buy-side (e.g. hedge funds) as quantitative portfolio managers. Our job is to try to identify ways to predict the future movement of securities. We have a harder job, but for the most successful quantitative traders, the payoffs can be huge. It’s much harder work than it used to be. Computing power has increased, data sets are commoditised, and now everyone knows the same tricks (even university students can try their ideas on Quantopian and there are videos that explain many of the core concepts of our craft ~ albeit naive!)

The quantitative investing revolution has largely been very successful, and quant strategies have been able to offer superior returns with lower risk than most other traditional investment strategies, which along with historically low interest rates, have allowed quant firms to become huge, collectively managing hundreds of billions of dollars of assets. But even in today’s crowded trading environment, unlike our predecessors in derivatives and algorithmic trading, we’re still not yet completely replaceable. Quants are still required to develop new research ideas to outsmart the market and other quants.

Different funds take varying approaches to the arms race. While funds such as the ultra-successful and ultra-secretive Renaissance Technologies continue to outperform by hiring relatively few (<100) elite researchers, others such as WorldQuant have taken the opposite approach, hiring teams of hundreds of mutually segregated quants working independently, allowing the firm to collect to the order of 100,000 alpha signals. Still other funds such as Quantopian take this to the extreme — and have attempted to completely crowd-source quantitative research (with questionable success).

While there are not many funds that publicly and provably claim to be successful in the use of machine learning in financial markets (unlike in tech, the best hedge funds usually try to keep a low profile), machine learning promises to allow funds to find new predictive relationships (“alpha signals”) automatically. Technologies such as reinforcement learning promise to take the process one step further by automating the actual trading decisions (“the strategy”). If and when machine learning finally fulfills its promise of conquering the financial markets, buy-side quants might discover that they have gradually automated themselves out of the research cycle and funds with large quant contingents may lose some of their economies of scale.

Capital vs Labour

The first Industrial Revolution allowed for capital to replace physical labour, reaching its peak in Britain in the tumultuous 1840’s and prompting Marx to write his Manifesto, forever transforming global society and political thought. Soon, AI researchers may enable capital (via GPUs and data centers) to replace intellectual labour. Who knows what sort of sociological tsunami that might bring about?

It’s all fun and games until the machines replace the intelligentsia

If three generations of quants’ experience in automating financial markets is anything to go by, the automation of rank-and-file AI practitioners across many industries is perhaps only a decade or so away. After that, a small group of elite AI practitioners will have made it to managerial or ownership status while the remaining are stuck in average paid jobs tasked with monitoring and maintaining their creations. Given the current level of the hype-cycle, the levels of pay given to AI researchers while they write the code that ultimately replaces them suggests to me that the relative power of capital versus labour is only getting worse. Add to this a continually increasing supply of STEM graduates from developing and post-communist-bloc countries, and it’s hard to imagine how short-term labour shortages in AI can continue for too much longer.

A career in sales anyone?

[1] https://www.nytimes.com/2018/04/19/technology/artificial-intelligence-salaries-openai.html