Waiting, waiting for the staging Hadoop cluster to finish a series of tests. My email chirped over the music in my headphones. I glanced at my inbox and saw an email from the version control system that was unusually large. One of the developers had been working alone for weeks on a project without a single commit, and here was all of it at once.

The new service dealt with using advertising views as a way of earning in-game currency. When a player wanted to upgrade their character they would have a choice of paying a dollar amount for the upgrade or viewing ads or offers to “earn” value. The company would show the ads or offers and once enough “ad value” was accumulated the company would pay the game company for the player’s in game credit. It was launched early Friday morning and by Sunday there were signs of problems. The money going in and coming out were not adding up. For $1 of ad views the company was paying for $1.25 in in-game currency. The developer who designed and wrote the system worked all weekend trying to find the problem with no luck.

I didn’t hear about the problem until after I came into the office Monday morning. When the symptoms were described to me I was reminded of what happened after I read through that single giant commit.

I still had time until my tests finished and looked at a few random files within the new project. A PHP script making database queries led to the database schema, which led to me taking off my headphones and looking around the room for the developer.

I found him and pulled him aside. “Hey man, using floating point numbers for dollar values is not a great idea. Floats can’t represent those values correctly”.

“It looks fine to me” was his response. The other developers kind of shrugged it off.

The source of the problem

Take this example in Objective-C:

NSLog(@"%", 1.0f - 0.41f);

This will print out:

0.590000

Which looks… OK. NSLog by default only shows 6 significant digits. If we tell NSLog to show 10 we see something else:

0.5900000334

Or 16 significant digits:

0.5900000333786011

The problem here is that floating point numbers cannot accurately represent base 10 (currency) values. Some tools or languages will hide the problem but since float , double , etc are all binary representations internally the problem is still there. Each time a mathematical operation is performed on them the “error” will grow. Pretty much all recent languages (and databases) have this behavior. This is described in much more detail in What Every Computer Scientist Should Know about Floating-Point Arithmetic.

The lesson here is never use a floating point type to represent a currency (decimal) value. In Cocoa use the NSDecimalNumber class to be safe.

Epilogue

By the end of Monday the company had lost over $150,000 due to the floating point problem. Far more in-game currency was purchased with the company’s real currency than should have been given the number of ad views. The company basically didn’t have that money and had to borrow more - and remove two employees. The developer who introduced the problem was not one of them.