fat fin·ger |fat 'fi NG gər |, verb, to enter a trade order incorrectly into an electronic stock trading system: He fat-fingered that sell order by accidentally adding two zeroes. The term comes from the idea of a clumsy, "fat-fingered" typist, who presses extra keys without being aware of it.

As the practice of high-frequency trading continues to become more widespread, concerns are growing that erroneous trades carried out by "algos gone wild"—a sort of digitally amplified version of the "fat finger" phenomenon—could cause a market crash at Internet speed, a meltdown that no one could stop. Two recent market glitches could provide a preview of what's to come.

Double-click of death: the Credit Suisse fiasco

Like Goldman, Sachs and other large banks, Credit Suisse has a proprietary trading desk—i.e., a division of the bank that trades stocks and derivatives in order to make money for the bank itself (instead of for the bank's customers). And, like other banks in its class, much of this proprietary trading is now done entirely by computers that execute trades by the millisecond based on tiny, fleeting fluctuations in a stock's price.

On November 14, 2007 at 3:20pm one of Credit Suisse's trading algorithms suddenly went haywire, and, in a few moments, sent hundreds of thousands of bogus requests to the exchange. This sudden surge of requests, which were cancellations for a large batch of orders that the machine had never actually sent out, acted like a denial-of-service attack on some parts of the New York Stock Exchange. The messages clogged the tubes and caused parts of the exchange to freeze up, affecting trading in 975 stocks.

After an extensive investigation, the NYSE assessed a $150,000 fine for Credit Suisse's "failing to adequately supervise the development, deployment and operation of a proprietary algorithm, including a failure to implement procedures to monitor certain modifications made to the algorithm."

The exchange's filing, released a little over a week ago, has the details of precisely what drove the algorithm haywire—it was a trader who accidentally double-clicked an icon in a trading program's interface, when he should've single-clicked. No, I am not making that up.

And you thought butterfly ballots were bad UI design

The algorithm that choked is part of a program called SmartWB, and it was designed to spit out a constant stream of trading orders all day long, using a set of fixed parameters. These orders would be transmitted to the NYSE, where they would go into a queue and wait to be executed.

A few days prior to the incident, a programmer took it upon himself to unilaterally improve SmartWB by adding a new user input feature, which would let a trader change a certain parameter on-the-fly. When a user changed the parameter, all the new orders that came out of SmartWB would reflect the change, and all of the existing orders in the queue waiting to be executed would instead be cancelled and replaced with otherwise identical orders that contained the tweaked parameter.

The interface for inputting this new parameter was really simple—and really, really stupid. There was a box for the trader to enter a number, and a pair of arrows next to the box: "up" and "down." Pressing the "up" arrow would revise the parameter upwards by the amount in the box, and pressing the "down" arrow would revise it downwards. After you clicked an arrow button, the new parameter would instantly take effect for all new orders, and all existing, queued orders would be cancelled and replaced with orders that reflected the change—no "Are you sure?" dialog box or any other form of sanity check, just instant execution.

The lack of feedback and "forgiveness" in the interface element would have been bad enough, but there was another problem, and one that would have been uncovered during testing... if there had been any testing, which there had not.

On November 14, a few seconds after 3:20, a trader put a number in the box and then double-clicked the "up" arrow. This double-click was interpreted by SmartWB as two separate clicks, so the system dutifully sent out a second batch of cancel/replace orders in addition to the batch that was intended by the trader. This sudden flood of cancel/replace orders, half of which were requesting cancellation of orders that had never been sent, overwhelmed the system and backed up five of the posts on the NYSE trading floor.

The incident wasn't a major catastrophe by any measure—it cost money, and overloaded a few posts near the close of the trading day, but otherwise it had no lasting effects. And the errors involved—a stray click, a bad UI decision, failures of testing and oversight—were all human errors, albeit amplified many times over by the speed and power of the network. You might even say that trader's the stray click was like the proverbial flutter of a butterfly's wing in China, except it didn't quite cause a hurricane in the Atlantic—just a nasty downpour.

Since 2007, our markets have moved drastically further in the direction of complete automation. But it's not clear that Wall Street's programmers have made correspondingly large leaps in testing, UI design, and version control. Indeed, on message boards and blogs, day traders who follow the market tick-by-tick swap stories of huge swings in a stock price, where a stock plunges or spikes but then corrects in a few minutes, after the exchange realizes there was an error and cancels the trades.

These glitches are typically attributed to high-frequency trading algorithms gone temporarily insane—"algos gone wild" is the preferred phrase. It is suspected that Rambus's stock was the victim of of an algorithm-driven selloff early this year.

The Rambus incident

On the afternoon of January 4, 2010, Rambus's stock lost 35 percent of its value in a matter of seconds. NASDAQ ended up canceling all trades at or below $20.73, and blamed the error on some unspecified trader's "fat finger."

Anecdotally, bizarre market action like this is becoming a fact of life for day traders, who now have to take extra measures to guard against being hurt by these moves. For instance, many day traders use automated "trailing stops" if they're going to be away from a terminal for a few hours—these stops dictate that if a stock drops by a certain amount, then the trader's platforms should put in an automated sell order to dump the security before it goes down even further.

If a trader was using an automated trailing stop on Rambus on January 4th, his system would've dumped the stock automatically, incurring a trading fee and possibly even losing money, only to see the stock jump back up again.

Of course, the fact that such computer-driven volatility hurts day traders matters little to long-term investors. But the fear is that these glitches are fleeting indications that the system as a whole is vulnerable and unstable, and that the right combination of circumstances could cause what happend to RMBS to happen on a wider scale. This is especially true as even more of the trading activity, even among individual traders, shifts to automated platforms.

Epilogue: From mainframe to PC

One trend of the past year that has been covered at the Financial Times and other outlets is the democratization of computer-automated trading. Small two- and three-person trading shops, manned by a laid-off quant or two and a programmer, are cropping up all over the country and combining the latest gaming-oriented GPU and CPU hardware with standard electronic trading accounts to do algorithmic trading on a small scale.

Because these groups are using commodity gaming hardware to generate "buy" and "sell" orders that are then transmitted to the market using standard retail trading software, there's no way to tell them apart from normal (human) day traders. Thus, there's absolutely no way to gauge how large of a phenomenon this actually is.

The Obama administration's efforts to rein in high-frequency trading by eliminating flash orders and banning proprietary trading (much of which is HFT-based) from large banks will probably have the effect of leveling the playing field a bit for these smaller algo shops. As Matthew Goldstein at points out in his Reuters article on the topic, the prop desks may disappear, but the software and expertise will not. Instead of being concentrated at a few large banks, algo trading will just spread, as the talent behind it either jumps to new funds or goes solo.

And if, thanks to further government intervention, the big boys ultimately lose some of the latency advantage that their deep pockets buy them, then the future of the market could well belong to hundreds of thousands of small trading operations who, instead of playing Crysis on their top-of-the-line Alienware systems, use them to print money instead.