Yesterday the New York Times published in print and online a rather bizarre article on GPS running watches, one that essentially concluded that they were an “unreliable running partner”. Now my goal isn’t meant to defend GPS watches, but rather clarify a lot of oddities and inconsistencies in the piece, and why I believe the New York Times missed the boat entirely.

Not all watches are created equal

The article starts off describing the basic features of most GPS watches in the market, such as displaying pace, distance and time – all of which represent the primary reasons that most people pickup a GPS enabled watch. From there the author goes into a singular test case where she and a friend met up on a run this past Sunday in the following scenario as a basis for much of the remainder of the article:

The challenge here is that I’d ask first – what were the watches in question? What brands, devices and software versions? For example, was the one that was off by .42 miles (97.3% accurate) an older model from 5-6 years ago, or was it more recent? Who made it? What firmware version?

In the world of GPS watches, the reality is that not all devices are created equal. As I’ve shown before in four posts of accuracy tests, some units do simply perform better than others. Sometimes that is correlated to price, and other times it’s tied to the GPS chipset used and/or the firmware. To base the entire article (and all GPS watches in general) on what appears to be a single watch on a single run being off seems a bit of a stretch. For example, when the Timex Global Trainer first came out, there were indeed accuracy issues with it. On average, it was 2.5% off (short) – was her watch a Global Trainer? Or perhaps, it was an original Garmin FR610 – which also had issues early on with some routes showing about 2% short. Yet, both have been fixed by their respective companies (June for the FR610, August for the Global Trainer).

I found it strange that the author didn’t note the brand, nor contact them for an official reason, explanation, or PR response. Isn’t that the most basic journalistic thing to do?

In my mind, this is no different than saying “cars are unreliable”, because your particular car is in the mechanics shop. As in fact the author noted, her friends route was just about spot on, within .08 miles after 19 miles – or 99.58% accurate.

And finally – there is of course the very real possibility that something might have simply been wrong with the unit, either temporarily or permanently. For example, on some older Garmin units, sometimes the unit will get stuck on a particular satellite set that will cause significant distance inaccuracies (i.e. a mile or more off). These are easily fixed through a soft reset, which clears the satellite cache database and forces the unit reacquire the satellite information upon starting up the next time. No electronics device is perfect, and while this isn’t common, it’s also easily fixed in two button presses and about 10 seconds of time.

Misunderstandings about race distances

The article then segues into a discussion on running race participants and having incorrect distances shown. It talks with one of the Race Directors for the Rock n’ Roll Marathons, regarding complaints that users often have about courses. This ultimately ended up with this snippet on accuracy:

This is actually a really interesting quote, but not for the reasons you’d think – and this is where the author shows a lack of research.

See, in a race scenario, it’s nearly impossible for a GPS unit to be short. The most common scenario is for a GPS distance to be long.

Why you ask?

Well, because GPS measures distance in a series of points, usually about 1-second apart. This means that with the exception of trail runs where you have a slew of switchbacks, it’s virtually unheard for GPS to be short…unless the course itself is short.

And in general when you have multiple people saying the course is short – it usually really is actually short.

See, most long distance road races with large crowds aren’t going to have switchbacks in them, and very rarely do they have a significant number of 180* turns (due to overcrowding) – turns that even if they missed a data point here or there, wouldn’t account for a third of a mile.

As I’ve gone into in (probably painful) detail in the past, when you’re running a big race with lots of folks, you usually end up running quite a few corners wide. And those corners add up. Remember that races are measured according to USATF standards and certified non-GPS devices, which require that the measuring person take the absolute shortest possible route during the measurement, right up to the edge of the curb. That’s not how the vast majority of folks run their races though. Instead, most folks are forced into much wider paths, often with swerving around other runners. Every time you swerve around a runner – you’ve probably added 5-10 feet to your path. But let’s look at a corner scenario:

During that post I noted that for the National Marathon course that year, there were over 64 roadway turns – that’s 64 opportunities to run wide. With each one likely being an extra 40+ feet of running! Even in the simplest example of the street above, with just two lanes, that’s half a mile of extra running!

One fun game to play is to visit Garmin Connect after a race, and search on the race title and date – then look at the results, sorted by distance (there are so many folks with Garmin devices, it’s virtually impossible to find a race without lots of uploaded data). Based on that, you’ll find the interesting pattern that typically the fastest runners at the front of the pack run the fastest lines (with less congestion), and thus have much closer accuracy rates – distances that are closer to the actual distance of the event.

Now, that’s not to say that there aren’t ways that the GPS device can malfunction and display a distance short – but most of these are edge cases. For example, even with tunnels – most tunnels are straight, thus the GPS unit will simply connect the entry points and exit points on the tunnel and measure the distance in a straight line between them.

And while tall buildings have tended to cause issues for older GPS units, newer units tend to have fewer issues here. One could certainly argue that products of years past should have been more accurate, but the reality is that technology does advance, and thus publishing an article about GPS technology without actually noting whose GPS technology is no different than saying broadly that cars don’t work, without saying which make and model you have.

Online Maps, Phones, and Tracks

Last but not least, the article uses Google Maps as a basis for comparison. At a high level, there’s nothing wrong with this, but at a more concrete and scientific level – mapping out your route on Google Maps just isn’t valid when comparing to a precise distance your unit recorded.

When you map out a route on Google Maps (or any other mapping provider) you’re usually following the street that you ran down, and then when you make turns, you follow the next street (typically on the white/yellow portion of the map that represents the street). But the reality is that most people don’t run to the middle of the street’s intersection to turn – they run on sidewalks, and each time you make a turn, you’re cutting that corner in comparison to the map you outlined. This is really then no different than the previous section on running corners long during races. Online maps also don’t typically account for the normal left and right ebb and flow of most running/biking paths, slight twists that over time really do add up.

That’s not to say services like MapMyRun aren’t useful – because they are, they’re incredibly useful. I used one Sunday for mapping out my long run, and again for Thursday’s long run. But not because I expected to get the measurement down to the hundredth of a mile, but because I got a rough approximate of my route within perhaps half a mile. Or enough to ensure that my planned 18 mile run wasn’t a 21 mile run (as my original route would have taken me on).

Next up is phones. While the article didn’t talk to it specifically, I do want to call it out just for fun. Running with phones has become incredibly popular, and for good reason. For many, a $1 or free app can do what a $200+ GPS running watch can do. But you really have to be careful in choosing your applications. Some of these apps do a tremendous amount of development around GPS data smoothing, to make your routes and workout distances more accurate. Data coming from GPS receivers in cell phones is notoriously unreliable, thus the significant work that folks like RunKeeper have to do to make it consistently usable. That’s one of the reasons why I generally recommend sticking to the major and most popular running apps when it comes to GPS recording – because those are usually the same apps that have the resources to invest in development fine tuning, which often goes on forever. If you look at the updates to the extremely popular RunKeeper application, virtually every update notes “GPS Accuracy Improvements”.

Finally, running tracks. The article focuses on tracks, based in part on two different sources, talking about inaccuracies on the track, this section highlights it best (the quote was from the Rock n’ Roll Race Director):

And, in fact, what they are saying at a high level is indeed largely true. GPS devices today do struggle to get down-to-the-meter accuracy levels on a track. The technology in consumer GPS devices just isn’t there today. Though, it is there in this system I tested back in September.

But it’s also important to note that depending on your recording settings (again, which vary by model/device and even different firmware versions on the same device), you can get some pretty darn curvy lookin’ ovals, such as a recent track run below:

The little edges that you see around the 8 and 7 markers are simply where I was grabbing a water bottle every few intervals during a rest portion while walking around. Otherwise, it seems rather oval-like to me.

That said, I still do believe that consumer GPS running devices can improve in the area of tracks. One idea I talked about during my ANT+ Symposium Keynote speech was the idea of a track mode, which would enable the GPS unit to understand what a track is and automatically have it ‘lock’ to the track, thus producing correct and exacting distances each lap.

Summary

The point here isn’t to say that GPS watches are perfect, because they aren’t (trust me, I’ve seen just about everything that can go wrong). However, they are generally quite reliable, for both training and racing.

When looking at GPS accuracy like the New York Times article did, it’s incredibly important to speak to exactly which units were used, and the circumstances around them. Sure, that may sound detailed for a newspaper article – but in the vast majority of other pieces I read from the New York Times (which I subscribe to) – I do get that level of detail. Whether it’s a piece about technology or politics in Eastern Kenya – that’s ultimately why people read the New York Times over an article on CNN.com; For comprehensive and thorough analysis. And that’s what disappointed me the most here, the lack of understanding of the subject, or the detail behind it.

As always, thanks for reading!