In-Depth

When Milliseconds Make Millions: Why Wall Street Programmers Earn the Big Bucks

David Ramel follows up his "Highest Paid Programmers" article with a look inside the world of high-frequency trading, where coders earn top dollars and even nanoseconds count.

A recent article I wrote about the highest paid programmers provided a peek into the world of big-money, automated trading on Wall Street. In high-frequency trading (HFT), programmers eke out every last incremental tick in performance to build algorithms that battle other algorithms for computational supremacy and millions in profits -- and earn a lot in the process.

So it wasn't surprising when, in response to the article, programmers on this site and others started talking about the tools, languages and techniques used in HFT in financial markets.

My previous article noted that an HFT company, Teza Technologies, offered $1.2 million to a C++ developer named Sergey Aleynikov to leave his $400,000-per-year position at Goldman Sachs and work on its own trading system. Aleynikov was convicted of illegally downloading proprietary code from Goldman Sachs' servers, which landed him in prison.

Aleynikov's trial provided a rare glimpse into the rarefied air of HFT -- sometimes known as algorithmic trading or black box trading-- which reportedly accounts for more than 70 percent of equity trading these days. It's an elite niche of computer science that few in the industry seemingly want to talk about in specific terms, probably because hundreds of millions of dollars are at risk. When the stakes are that high, the players keep their cards pretty close to their vests. As an AdvancedTrading.com article described HFT firms, "They are, as a rule, secretive, stealthy, smart and relatively unknown." In fact, the presiding judge in the Aleynikov trial sealed the courtroom to hear testimony about the proprietary HFT source code in question.

There are many different aspects to HFT, but basically it comes down to building algorithms and computer systems that can monitor and digest huge amounts of financial data in order to automatically conduct transactions in much less time than it would take a human trader to glance from his BlackBerry to his Bloomberg Terminal. Or, as Wikipedia explains:

"In [HFT], programs analyze market data to capture trading opportunities that may open up for only a fraction of a second to several hours. HFT uses computer programs and sometimes specialized hardware to hold short-term positions in equities, options, futures, ETFs, currencies and other financial instruments that possess electronic trading capability. High-frequency traders compete on a basis of speed with other high-frequency traders, not long-term investors."

HFT was blamed for the huge drop in the stock market in May 2010, called the "flash crash." The Wall Street Journal reported that, "At one point, HFTs traded more than 27,000 contracts in just 14 seconds."

In this world, speed kills -- or lets your program live another day, if it's working well and beating other systems. Tackling latency is of primary importance, with programmers using every last tool and trick in the book -- and continually making new ones -- to squeeze out every last millisecond of performance.

Or would you believe every last nanosecond, even?

A nanosecond is 1 billionth of a second. One company, Fixnetix, recently announced that its "microchip for ultra-low latency trading is now the world's fastest trading appliance for the financial markets. Customers are now seeing latencies as low as 740 nanoseconds through the stack (wire-to-wire)."

So what kind of language tools and techniques are programmers using to develop such high-performance systems? I suspected it would be something as close to the metal as possible, perhaps even assembly language for certain parts or some esoteric proprietary language. But some surprises emerged in readers' discussions of the previous article, with languages like Haskell and Smalltalk mentioned prominently.

However, it's no coincidence that Aleynikov was a C++ guy. As HFT expert Matt Davey, chief technology officer at Lab49, a financial services industry consulting firm, explained in an interview: "From a HFT platform perspective, C/C++ is the language of choice due to the latency requirements," he said. "The lower the latency, the more C/C++ is important."

A contract programmer for a New York City firm doing "trading strategy back testing" I talked with who didn't want to be identified confirmed this. "I work mostly in C++," said "Jack" (a pseudonym), "but also Java, Lisp, Matlab, R, Fortran and JavaScript."

Programmer/entrepreneur Jason Roberts told me the languages/tools used by HFT coders "really varies from firm to firm. Some use the Microsoft stack (.NET, C#, etc.), some use Java or C++ and others use more exotic languages like Ocaml."

Roberts has a unique perspective on the HFT industry, having worked as a founding partner for a series of small, proprietary trading companies and for a hedge fund, developing the trading technology. "I built everything from the order managers and feed parsers to the low latency messaging systems, trading UIs and tick capture databases, machine learning libraries and trading engines," Roberts said. "In most cases, I developed everything (or most everything) entirely myself."

Surprisingly, Roberts said he used C++ sparingly. "I never really used anything very esoteric and, in fact, tended to rely on tools that were fairly common and unsophisticated. For example, back between 2001 and 2003, I wrote two entire HFT trading systems (one for option market making and the other for equity pairs trading) using VB6, which even back then would've struck many as silly. The reality was that the applications worked, they were reliable and they were fast (since VB6 is compiled)."

"Moreover," he continued, "I was very productive using VB6, which enabled me to get a lot more done than if I had attempted to build everything using C++. In fact, I would only use C++ for the very rare algorithm that needed an extra boost of speed, and the one I remember best as being slow in VB6 was the square root function, of all things!"

Mike O'Hara, publisher of the High Frequency Trading Review, also listed the "C" languages as being prominent in the industry, along with Java, Matlab and Cuda. The latter, he said in an interview, is used for programming GPUs for an extra boost in performance, which seems to be an increasing trend (more on that later).