When Facebook announced in 2016 its plans to outsource fact-checking to a group of third-party specialists, even some critics gave the company credit for trying to help solve the misinformation or fake news problem. But in its efforts to appear as politically neutral as possible, Facebook set a trap for itself, and that trap was sprung last week, when the conservative news magazine The Weekly Standard flagged as false a piece the left-leaning ThinkProgress wrote about Supreme Court judicial nominee Brett Kavanaugh and his views on abortion.

The ThinkProgress piece argued that Kavanaugh’s testimony during his hearing made it clear he would likely vote to overturn Roe v. Wade, the landmark case that made abortion legal in the US. While the piece itself was fairly nuanced, the headline was not: It said “Brett Kavanaugh said he would kill Roe v. Wade last week and almost no one noticed.” The Weekly Standard‘s fact checker, Holmes Lybrand, flagged it as false (as he pointed out on the site) because Kavanagh never actually said that.

ICYMI: Vanity Fair criticized for September cover shoot

This set off howls of outrage from a number of corners, including ThinkProgress founder and former Editor in Chief Judd Legum, who suggested the incident called into question Facebook’s entire fact-checking program, and was another sign the social network was bending over backwards to show it wasn’t biased against conservatives.

1. Facebook's "fact checking" system is a farce. It's purpose is not to check facts — it's to appease the right wing. That's why they made THE WEEKLY STANDARD, a partisan right-wing magazine and official fact checker Sign up for CJR 's daily email It's going very poorly — Judd Legum (@JuddLegum) September 11, 2018

In solidarity, The Intercept ran the disputed ThinkProgress piece with a somewhat over-the-top editor’s note, claiming the story had been “effectively nuked,” since links flagged as false tend to lose as much as 80 percent of their traffic on Facebook. The Intercept folks could probably have saved their breath, however: Even after the story was flagged, it still racked up about three times as much traffic as the average ThinkProgress piece, according to data from Facebook-owned CrowdTangle.

Facebook handing its fact-checking program over to a conservative website so it can nuke whatever progressive stories it disagrees with makes for a great story—Vox entitled its explainer “Facebook blocked the spread of a liberal article because a conservative told it to”—but the truth isn’t quite as simple.

No one knows this better than Alexios Mantzarlis, who runs the International Fact-Checking Network, which is based at the Poynter Institute. The IFCN was set up to develop (and enforce) standards for fact-checking, not just on Facebook but anywhere. After Facebook launched its program, it looked to the IFCN to tell it which fact-checking services were reputable and which weren’t, based on the network’s published criteria.

In an interview, Mantzarlis said the IFCN doesn’t verify the accuracy of everything an outlet publishes, it only checks to see whether the fact-checkers who work there follow the standards it sets out. And according to that yardstick, The Weekly Standard passed all the appropriate tests. The site has fact-checked over 120 articles, and Lybrand limits his work to fact-checking; he write neither news nor opinion.

“I’m not denying The Weekly Standard is a partisan publication, but the decision we made was that we would look at the partisanship of the fact-checking operation itself, not the entire publication,” Mantzarlis says. “We think it’s better to have a broad tent. And we had to do this in light of some verifying organizations around the world, like Liberation in France, which is a clear left-of-center newspaper but also one of the oldest fact-checking operations in the world.”

According to Legum, The Weekly Standard was chosen as a fact-checking partner by Facebook specifically because it was conservative-leaning, and then the IFCN was forced to validate its fact-checking operation (it passed after failing its initial validation check). In a post for his morning newsletter, Legum quotes the IFCN evaluator, journalism professor Mark Coddington, as saying the site failed to meet a number of tests, including the length of time it had been doing fact checks.

Mantzarlis says The Weekly Standard was actually recommended by Facebook, which wanted to make sure the fact-checking program was politically balanced. One source says Facebook executives saw a story about The Weekly Standard in which Editor Stephen Hayes made a point of saying he wanted the site to be a “fact-based” alternative.

“With the rise of non-credible, clickbaity outlets on the right, we thought there was a role to play in helping conservative news consumers separate fact from fiction,” Hayes told CJR in an email. “Beyond that, over the years, we’ve published a number of articles critical of left-leaning fact-checkers. We thought it better to join the process and formalize our fact-checking program than simply complain about them from the sidelines.”

The Weekly Standard was initially refused by the IFCN, Mantzarlis says, but then it made a number of changes that addressed the evaluator’s concerns, including where and how it would run its fact checks. Once that was done, he says, the site was admitted as a partner, and the amount of time it had been doing fact-checking was reduced (from the standard three months) because of what Mantzarlis says was a miscommunication. However, he notes that Washington Post writer Glenn Kessler and Politifact are both members of the IFCN approval board, and neither would be likely to bow to pressure from either Facebook or the right.

“The idea that a board composed of five non-American fact-checkers and two American fact checkers who have been on the receiving end of pointed barbs by The Weekly Standard’s editorial pages for years somehow gave the organization preferential treatment is absurd,” Mantzarlis says.

Mantzarlis admits, however, the Kavanaugh hearing story by ThinkProgress shows some flaws in the program. For example, the headline may have been inaccurate, but the piece itself made a specific point and backed it up with evidence. Flagging the whole thing as false isn’t really fair. And yet, as Mantzarlis noted in a piece for Poynter analyzing what happened, fake news stories often spread in part because of their headlines, since few people read any further.

Think Progress is understandably peeved. There's a case to be made that Facebook's tool should not be used to downrank analysis. There's also a case to be made that saying Kavanaugh "said" something can be understood as meaning that he literally said it. Which Kavanaugh did not — Alexios (@Mantzarlis) September 11, 2018

Should there be a separate flag for headlines? Perhaps, Mantzarlis says. But what The Weekly Standard brouhaha reinforces more than anything is that the black-and-white nature of Facebook’s fact-checking program is a blunt instrument, and the problem of misinformation on the platform (and elsewhere) has a lot more nuance.

“The challenge is Facebook has yet to decide what the hell its fact-checking product is for,” Mantzarlis pointed out in a Twitter thread arguing the program needs more work. “Is it to clean up the junky viral hoaxes about sharks swimming up interstates? Or to target inaccurate information in all its guises? FB has defaulted to the latter interpretation because it doesn’t want to make an editorial judgment about types of misinformation to prioritize.”

As Claire Wardle of the Shorenstein Center’s First Draft project has noted, the term “fake news” can be applied to a broad range of misinformation, from parody and harmless mistakes to hyperbole and deliberate attempts at political propaganda distributed by paid government agents. A binary flag is never going to be able to capture all of that, which means it is a way for Facebook to appear to do something without having to tangle with the messy reality of misinformation.

A Facebook spokesman denies the social network forced the IFCN to accept The Weekly Standard, and says the company relies on the IFCN to decide who gets accepted into the program and what standards they have to meet. The spokesman says Facebook is “thinking about making some improvements to the program, but there are no specifics to announce right now.”

ICYMI: Facebook’s troublesome local media tactics

Has America ever needed a media watchdog more than now? Help us by joining CJR today

Mathew Ingram is CJR’s chief digital writer. Previously, he was a senior writer with Fortune magazine. He has written about the intersection between media and technology since the earliest days of the commercial internet. His writing has been published in the Washington Post and the Financial Times as well as by Reuters and Bloomberg.