Is Google broken? Or is your site broken? That’s the question any webmaster asks when she sees her Google click-throughs drop dramatically. It’s a question that Matt Haughey, founder of legendary Internet forum MetaFilter, has been asking himself for the last year and a half, as declining ad revenues have forced the long-running site to lay off several of its staff. MetaFilter, Haughey told me, made the bulk of its revenue through Google’s AdSense program. That is, MetaFilter worked with Google ad representatives to place advertising on its pages that was served by Google. In turn, most of MetaFilter’s visitors came via Google searches. (MetaFilter’s members pay a small one-time signup fee to see no advertising; I am a member but not a terribly active one. I should also disclose here that I used to work for Google and my wife still does.)

Founded in 1999, MetaFilter, or MeFi, quickly earned a reputation as one of the most intelligent and civil discussion sites on the Web. A core group of sharp, technically savvy users discussed culture high and low, while a small group of moderators walked a careful line between policing abuse and allowing free expression. Early users included some of the key movers in blog culture, including Anil Dash, Jason Kottke, and Meg Hourihan. It served as the model for the best-moderated forums on Reddit, where actual content outweighs noise, trolling, and link spam. Its subsite Ask MetaFilter was one of the best places to ask questions on all subjects and get intelligent answers, from “What clever relationship ‘hacks’ have you come up with?” to “What have you learned through your career, major, or specialization that you wish the general public knew?” to “If you killed somebody, how would you dispose of the body without getting caught?” If, like many Slate readers, you’re considering a septum piercing, MetaFilter’s page on pros and cons is far more informative (and better-spelled) than Yahoo Answers’ or Body Jewellery [sic] Shop’s (both of which Google ranks above MetaFilter if you search on “septum piercing pros and cons”).

In short, MetaFilter is the sort of site that makes the Web better. But in October of 2012, something terrible happened, as recounted by Haughey. The something terrible is in this graph, which shows MetaFilter site visits across time:

Traffic suddenly dropped by 40 percent, and stayed there. The graph shows visitors on a day-by-day basis, which follows a consistent weekly pattern until it nearly drops by half toward the end of 2012. With fewer visitors came fewer ad click-throughs, and within weeks MetaFilter’s ad revenue stream had nearly been halved. According to Haughey, the drop in traffic owed almost totally to drops in click-throughs from Google search results. MetaFilter had not made any particular changes to trigger this drop, and Google, as usual, wasn’t forthcoming about what might have caused it. Haughey tried reducing the number of advertisements, wondering if perhaps MetaFilter had too many ads, since Google had been known to penalize sites for being too ad-heavy. But that didn’t help. His Google AdSense representative had no answers, Haughey said, other than to ask if he was interested in running more ads by Google.

The drop in Google traffic to MetaFilter almost certainly owes to MetaFilter pages appearing lower across the board in Google search result pages. To understand a bit about how search result ranking affects website revenues, you have to understand the so-called Clickthrough-Rate Curve, which shows how frequently users click on the first search result, second result, and so on. According to ad network Chitika, you’d better be in the first few results if you want anyone to see your page:

Fully one-third of all clicks resulting from a Google search are on the top result. The second and third eat up nearly another third. People look at the second page of search results less than 10 percent of the time. These results may be conservative: A 2012 study suggested that the first link gets clicked on over half the time, while another study suggested roughly 40 percent. This is why search engine optimization, or SEO, is so vitally important and such a moneymaker.

Google, of course, does battle with “black hat” SEO organizations that try to trick its algorithms, continually altering its ranking system to try to reward pages that play fair while penalizing those that don’t. “Link spam” was one of the most notorious ways SEOs tried to uprank their sites, by plastering spam links all over other people’s sites. Part of Google’s algorithmic updates in 2012 was specifically designed to address this webspam problem.

But the people behind MetaFilter weren’t perpetrating any of these SEO crimes; Haughey said that as far as he knew, they were committing no violations and were working hard to follow Google’s recommendations. Still, they clearly got caught in the 2012 shakeup.

Google is very much a black box when it comes to ranking. Google issues general guidelines—remove user-generated spam, keep links to a “reasonable number,” create a “useful, information-rich site,” don’t overdo guest blogging, and so on—but as long as you aren’t blatantly trying to game Google, it can be difficult to know exactly why one site beats another in the rankings. While I can’t answer the question of why MetaFilter’s pages dropped in Google’s search results, I can ask a different question, which is whether the drop is justified.

I took a look at roughly 100 MetaFilter pages, most of them on its Ask MetaFilter discussion section, and came up with search queries that I thought would lead to them. For reference, I compared the results against results from Bing (which also powers Yahoo and partly powers DuckDuckGo).

Looking at the results, I can say a few things with fair confidence. First, MetaFilter is not being penalized across the board. It had appropriately high-ranking pages for a number of my queries. Second, for a decent number of pages, Bing and Google treated MetaFilter roughly equally. Sometimes Google ranked MetaFilter a little higher, sometimes a little lower, but there was nothing to raise an eyebrow. But third—and this is the important point—there were a nontrivial number of queries for which Google ranked MetaFilter pages way lower than Bing did, and seemingly without good reason. It was much rarer to see Google rate MetaFilter way higher than Bing did.

For example, if you search “maybe there is a god,” Bing returns, in the No. 2 spot, a MetaFilter page with agnostic book recommendations, titled “Books about ‘maybe there is a God’ for an agnostic with some doubts.” Fair enough. On Google, though, the MetaFilter page doesn’t show up until the sixth page as the 60th result. (Search engines make tiny algorithmic alterations all the time, so placement may be approximate.)



Or consider the query “typology of joy” (not in quotation marks). Bing aptly returns MetaFilter’s page—“A typology of joyful pursuits?”—as the top result. Google’s top result is a page called “Topical Bible: Types of Christ: Paschal Lamb.” The MetaFilter page does not show up until halfway down the second page of results.

One last one: Searching for “most amazing woman ever” on Bing will give you MetaFilter’s helpful “Who is the most amazing woman who ever lived” as the third result. (Answers, by the way, included British spy and French Resistance leader Nancy Wake, world’s first programmer Ada Lovelace, slave rescuer and activist Harriet Tubman, and Chinese pirate Ching Shih.) Google puts it at the bottom of the second page of results, in 19th place. Google’s top result? A list of “the 100 most beautiful women ever.”

I turned up a number of other examples of this sort of puzzling deranking of MetaFilter pages, in which I judged the deranking to be unjustified, with many worse results above. Again, this deranking (i.e., lowering of the rank of a site) did not happen across the board, but it happened surprisingly often across the hundred or so queries I tested. My data is firmly anecdotal, due to a lack of time and resources on my part—I wouldn’t dare claim that Bing is better than Google, and some of the derankings may just be the luck of the draw. As a site that tries to attract users with solid content rather than flashy headlines and Share This buttons, MetaFilter may simply be getting caught between social linkbait sites like Upworthy, prestige properties like Yahoo Answers, and ultra-SEO’d sites. But the combination of the abrupt 2012 drop in MetaFilter’s fortunes and the preponderance of derankings leads me to suspect that something fishy is going on in the deep recesses of Google’s alchemical ranking algorithms, and that it deserves investigation. I doubt Google is intentionally penalizing MetaFilter—could it be a bug?

If so, it’s a shame: MetaFilter is one of the best-moderated sites on the Internet, and it’s tragic if it’s losing revenue to mistaken deranking, when linkbait and song-lyric sites frequently clog up the higher rankings. But all I have are suspicions. Will the Google oracle answer them?