As an example, let’s take two articles from roughly the same time period of Haile’s article: The Dark Power of Fraternities, The Atlantic’s cover story for their March 2014 issue, and Here’s Ellen’s Mid-Oscars Selfie with All of the Stars, an article from The Wire on March 2, 2014. It’s hard to imagine two articles that differ more substantially. One is a serious investigation on how fraternities impact university culture; the other is quite simply a picture taken by celebrities at the Academy Awards. Set aside the obvious editorial differences: the articles also differ greatly in article length.

The fraternities article is the very definition of long-form; created in The Atlantic’s new feature template, it requires some measure of time to read and understand. On the other hand, The Wire’s article can easily be digested and shared in a matter of seconds. According to the reading time model, because you didn’t linger on The Wire’s article for several minutes, it may not qualify as a quality engagement and the resulting shares are simply a byproduct of visitors perhaps sharing the headline rather than the content. I say that’s a rather limited view of what constitutes “engagement” for the modern media consumer, as well what is valuable on a publisher-by-publisher basis. Not every visitor needs an in-depth analysis of Greek life in American universities; sometimes they just need a quick download of the latest pop culture news. Yes, even if it’s just a selfie.

We shouldn’t assume that all visitors share the same preferences.

One person coming to read an article might be doing so from home, while another person visits during their commute. Visitors might not even be using the same type of device to access a website, a key function of today’s multi-platform technological landscape. One of my colleagues at Atlantic Media Strategies, Zach Kalman, rightly takes issue with another of Haile’s claims that “on the native ad content we analyzed, only 24 percent of visitors scrolled down the page at all, compared with 71 percent for normal content.”

Kalman asks, “How does a statement like this control for device type, screen size and even site design?”

Any article is going to have significantly different layout on mobile versus desktop, particularly in the age of responsive design. What applies to desktop computers cannot possibly be meaningful for mobile phone and tablet devices except in the broadest possible generalization.

I immediately think of a site such as Quartz that has a reasonably mobile audience, and where the vast majority of article discovery is done via infinite scroll, and it makes me wonder if it could possibly draw any similar conclusions with Chartbeat’s findings. Does Time, which now features scroll as a primary discovery mechanism, see the same results that Haile describes? I doubt it.

This is where individual metrics fail, they are each alone applied as panaceas to solve all of media’s problems.

First the hit, then the page view, and now reading time.