When we reviewed Google and LG's new Nexus 5 earlier this month, we noticed that while it outscored other phones with the same chip in our standard benchmark tests, its behavior wasn't always consistent. Typically, phones have no problem making it through the 30-or-so minutes of CPU, GPU, and storage testing that we perform. We'll run most of our benchmarks multiple times to check for outliers, and nine times out of ten the scores will come back within spitting distance of each other.

The Nexus 5 presented outliers—scores in the Geekbench CPU benchmark, for example, could come in around a third lower than the typical score depending on how many times we ran it or what we had been using the phone for beforehand. When we reviewed the Nexus 4 last year, we noticed that it had similar problems, throttling its CPU speed aggressively to prevent overheating.

We decided to take a closer look at the Nexus 5's CPU throttling and to explore the concept of mobile SoC throttling in a bit more detail—there are times it will matter and times that it won't, but it's a common enough phenomenon that it's worth investigation.

Throttling

Let's be clear: most mobile devices do some kind of performance throttling when they're under a lot of stress. It's basically unavoidable—most smartphones and tablet SoCs are built to run very quickly for a short amount of time, but under stress they need to tone it down to prevent melting whatever enclosure they're in.

When we noticed the Nexus 4 throttling in our benchmarks last year, it wasn't necessarily notable because it was happening, but because it was happening so quickly and so often. If you look at the Apple A7 in the iPhone 5S, you'll see a chip that can run at its max rated CPU frequency of 1.3GHz for about two minutes before coming down to about 1GHz. The Nexus 4 couldn't even make it through our entire benchmark suite (though it's worth noting that later hardware revisions improved this somewhat—a Rev 1.1 Nexus 4 purchased in May throttled less and ran cooler than our Rev 1.0 Nexus 4 from last November, as we'll see in a moment).

There are good reasons to throttle your SoC, but it's a delicate balance. Throttle too much, and users won't see the benefits of that fast SoC they paid for. Don't throttle aggressively enough, and heat and battery life will both suffer. Typically, the CPU will start out at its maximum rated speed and fall off as the phone gets warmer.

To see how the phones performed under extreme stress, we grabbed Intostudios' Stability Test app from the Google Play store. This app will slam all of your phone or tablet's CPU cores continuously, measure how many times the phone completes a single test cycle, and report your phone's internal battery temperature. Using the excellent System Monitor from Christian Gölinger, we turned on a background monitoring service to monitor CPU clock speed over time. We compared the Nexus 5 to Samsung's Galaxy Note 3, another recent phone with a Snapdragon 800 SoC.

Looking at the Nexus 5 and the Note in the first five minutes of testing, the first thing you'll notice is that the Note spends more time running at its max rated clock speed of 2.3GHz. Both CPUs' clock speeds bounce up and down as the test goes on and more heat is generated, but the Nexus 5 drops off much more quickly and can't even crack 2.0GHz by the time the test has been running for a minute. The Note 3 will go down to a lower clock speed, but it will also go back up to 2.3GHz more regularly, resulting in better overall speed. If you continue running the test, both phones begin to throttle even more as they heat up. But the trend is the same: the Note 3 runs faster for longer.

Phone SoC Cycles completed in one hour Nexus 5 2.26GHz Snapdragon 800 320 Galaxy Note 3 2.26GHz Snapdragon 800 372 Nexus 4 (Rev 1.0) 1.5GHz Snapdragon S4 Pro 296 Nexus 4 (Rev 1.1) 1.5GHz Snapdragon S4 Pro 360

By the time the test had been running for an hour, a fairly wide gap had opened up between the Nexus 5 and the Note 3—the latter completed many more test cycles in the same amount of time. The difference between the launch Nexus 4 and the more recent revision is also striking. The older phone eventually got so hot that one of its CPU cores just completely shut off, driving the temperature down from the low 60s (Celsius) to the mid-to-high 40s but greatly affecting the amount of work the ostensibly identical processor can get done.

The newer Nexus 4 managed to keep all of its CPU cores engaged for the duration of the test, and in fact it completed more cycles in an hour than the Nexus 5 did (slow and steady apparently wins the race). The Nexus 5 begins much faster but can't maintain that speed advantage over a long period of time.

When does it matter?

The test above isn't necessarily indicative of anything that you'll notice as you actually use these phones. Real applications that will max out all four CPU cores are rare; applications that will do it for a solid hour are essentially nonexistent. The real apps most likely to push your phone's thermal limits for that long are games—they'll hit the CPU and GPU hard enough for long enough that you might start seeing some real-world slowdown if you play for long enough.

That said, it's difficult to find real-world tasks that will actually trigger serious throttling, at least not to the extent that it degrades the user experience. Playing a reasonably intense game like Need for Speed Most Wanted with the CPU monitor open shows that the two active cores do slow themselves down after a few minutes of play, but not to the extent that the frame rate or responsiveness suffers. As games catch up to the Snapdragon 800 that may not always be the case, but at least for now, the game runs as well (and as long) on the Nexus 5 as it does on the Note 3.

We did notice a significant difference in performance when encoding video—it's a bit of an edge case because your computer's hardware and software are much, much better suited to this task, but like the stress test, media encoding uses a lot of CPU resources for a sustained period of time (we used the Video Convertor Android app, which is a design nightmare but will work well enough for our purposes).

Using the default conversion settings, we observed that both phones began converting our test video (Louis CK's hilarious Oh My God) at roughly the same rate. Once a few minutes had elapsed, the CPUs throttled down—the Note 3 settled into converting video at the rate of about 23 FPS, while the Nexus 5 could sustain just 16 FPS. This is nearly a third slower, and you'll definitely notice the difference if you're converting a large video. In fact, it's not much higher than the 15 FPS the rev 1.1 Nexus 4 could sustain, though in the end the conversion will still be faster because the Nexus 5 starts off faster.

When it doesn't matter, and what to take away

Thankfully, most smartphone usage doesn't involve converting videos, so much of the time you won't be able to notice the differences between the Nexus 5 and something like a Galaxy Note 3 with less of a propensity for throttling. Most device usage is "bursty" in nature—short periods of activity followed by longer periods of idleness. Processors have been taking advantage of this for years to save power, scaling their clock speeds, core counts, and voltages down instead of running full-tilt all the time. These patterns of activity are the ones that the better benchmarks are actually pretty good at simulating, and the Nexus 5 stands at the top of those charts for a good reason. Most of the time, its Snapdragon 800 feels the same as any other silicon.

Here's what you should take away, especially if you're coming from the desktop and laptop world where performance between devices with the same chip is more consistent and predictable: in the thermally constrained world of smartphones and tablets, a 2.26GHz Snapdragon 800 in one device isn't always going to give you the same performance as the same chip in a second device. Actual performance will depend on a variety of factors, including the size and design of the enclosure, the way the OEM has tuned its phone or tablet, and the particular tasks that you're using your hardware for.

The other big takeaway is that advertised CPU clock speeds aren't always directly comparable—a Snapdragon 800 sold as running at 2.26GHz isn't the same as an Intel Core i5-4200U sold as running at 1.6GHz. Intel gives you the sustained clock speed for its desktop and laptop chips, but it also provides a separate faster Turbo Boost speed (in this case, 2.6GHz). The Snapdragon 800 only advertises its Turbo Boost-equivalent maximum clock speed, but not the speed that chip will run at under sustained load (perhaps in part because it varies so much). For what it's worth, Intel is edging closer to Qualcomm's strategy with some of its most recent mobile chips, possibly because it doesn't want to be seen as lagging behind in any kind of clock speed war. The company lists both a 1.33GHz standard clock speed and a faster 1.8GHz "burst frequency" for things like its Bay Trail-based Atom Z3740D, but the burst frequency speed is the one that's advertised up-front for products like Dell's Venue 8 Pro tablet.

Finally, depending on the OEM, it's possible that devices from early in the manufacturing run will have heat and throttling characteristics that are very different from devices later in the run. The differences between the rev 1.0 Nexus 4 and the rev 1.1 model when under load are significant, as much or more so than the jump between the rev 1.1 Nexus 4 and the new Nexus 5. There's no guarantee that this will happen with every phone, but it's possible that a Nexus 5 bought six months from now will be a little better-behaved than a Nexus 5 bought at launch.

Throttling is totally normal, and it's necessary if you want to create a single chip for both phones and tablets (as both Qualcomm and Apple are doing with the latest Snapdragons and the A7, respectively). None of our findings should be interpreted as scandalous or abnormal. It just happens to be the case that among Snapdragon 800 phones, the Nexus 5 throttles its clock speeds aggressively. Especially over the course of a couple of years as software becomes more demanding, the differences in speed between it and other Snapdragon 800 phones may become more apparent to users.