Rank Hotness With Newton’s Law of Cooling

By Evan Miller

February 15, 2009

Greg A. writes,

I enjoyed your post on how to not sort by average rating. I was wondering if you have any experience with sorting based on average votes over time to get a hotness rating.

Greg, I assume you mean “hotness” in the Reddit sense rather than the HotOrNot sense.

Fig. 1: what Greg is talking about Fig. 2: not what Greg is talking about

Neither concept of hotness is defined with any rigor, but I imagine a “what’s hot” list should show what items, discussions, or products have gotten a lot of recent activity. I’m not going to analyze all the hotness algorithms out there, just my favorite.

I recommend a technique called exponential decay. Exponential decay has three components:

Each new item has an initial “temperature” reflecting its hotness The temperature is increased by a fixed amount every time someone gives the item a thumbs-up The temperature gradually drops down over time

A hot list then sorts items by temperature. As we’ll see, exponential decay is great because:

It’s easy to understand It goes easy on the database It can work without modification as your site gets more popular

First let’s talk more about the mechanics of exponential decay. Bumping up an item’s temperature in response to new activity is the easy part; but how do you figure out how much it has cooled? You first need to decide on a cooling rate, that is, how many hours it should take for the temperature T to fall by roughly half. With that in hand, you calculate:

(Current T) = (Last T) × exp( −(Cooling rate) × (Hours since last T) )

(The exp function means take Euler’s number e=2.71828… to a power.)

The nice thing about this method is that you only need to write to the database when you’re incrementing the temperature. The rest of the time, for example when you’re sorting a list of items to display, you can calculate every item’s current temperature just based on its last reading, no matter how many hours ago. That is, you don’t need to constantly update the database, and you never need to process the rating history!

If you play with the decay parameters a bit, you should be able to show the right mix of new and hot items at the top of your list. With a small initial temperature, you’ll see fewer brand-new items; with a smaller decay rate, popular items will tend to linger longer.

Graphically, here’s how the temperature will drop over time. The shape of the curve is always the same for exponential decay; the parameters only determine the scale of the axes.

The choice of parameters is ultimately be a judgment call (or not; ask me sometime about A/B testing). As your site gets more popular, you may want to tweak the parameters, but the same parameters should work fine over a large range of site activity (i.e., no matter whether the top item has a temperature of 10 degrees or 10,000 degrees). Because the decay is exponential rather than quadratic or linear, even extremely popular items won’t get “stuck” at the top for too long after their popularity has peaked.

So in a nutshell, you will:

Pick an initial temperature for new items Pick a cooling rate Pick a temperature increment When there is new activity on an item, calculate the current temperature, then increment and record it along with the current time Sort items based on the current temperature using the formula above

And voila, you’ve got yourself a “hot” list. Another good use of this algorithm would be listing active discussion threads in an online forum (where each reply increases the temperature).

One last note about exponential decay: besides being easy to compute, it’s also the law that governs how any hot item cools down, from a hot brick to a hot tamale. It’s called Newton’s Law of Cooling.

REFERENCES

Exponential decay (Wikipedia)

Newton’s Law of Cooling (Wikipedia)

You’re reading evanmiller.org, a random collection of math, tech, and musings. If you liked this you might also enjoy: Deriving the Reddit Formula

How Not To Sort By Average Rating

Bayesian Average Ratings

Get new articles as they’re published, via Twitter or RSS.

Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!



Wizard

Statistics the Mac way

Back to Evan Miller’s home page – Subscribe to RSS – Twitter – YouTube