“Facebook has more power in determining who can speak and who can be heard around the globe than any Supreme Court justice, any king or any president.” Those prescient words came from law professor Jeffrey Rosen way back in 2010. Five years later, the Times is willingly handing its censorship keys over to that king of kings.

Much has been made of Facebook’s potential new partnership with the Times, Buzzfeed, and a handful of other news organizations, who may soon start posting stories directly on Facebook instead of having Facebook readers reach their content through a link. This move has the potential to make a lot of money for cash-strapped news organizations and produce another anchor into the news world for the cash-flush social network.

It also has the potential to rob news organizations of their soul. Felix Salmon believes this could kill the news brand (it could). Others, like Mathew Ingram, argue that it could give Facebook too much control over which news organizations thrive and which will die when the social media company decides to tweak its algorithm (it does). But the problem is much broader than that.

What this discussion has missed is perhaps the most crucial element of Facebook’s new power: the right to choose between the free expression of ideas or to instead impose censorship when it deems content unworthy. That should worry the public, because when given that power in the past, Facebook has ruled with an iron fist.

The New Yorker was famously banned from Facebook for a short period in 2012 for posting a cartoon with a tiny bit of nudity in it. Breastfeeding photos have been the source of takedowns and controversy. And that’s even when they’re not censoring other photographs or news commentary by “mistake.”

Then there are the hostage negotiations country’s governments are increasingly engaging in. Turkey blocked Facebook and Twitter across the entire country just this week because users were sharing a photo that was clearly newsworthy, and vowed to continue blocking it unless the social networks removed it (which they did). Two weeks after the Charlie Hebdo attacks, Facebook began censoring images of prophet Muhammad after Turkey made yet more demands.

Sign up for CJR 's daily email

And what happens when Mark Zuckerberg brings Facebook to China? Will Facebook be just as quick to censor the New York Times posts at the whims of foreign governments as it does for individual users?

Facebook is extremely meticulous about what content the public should see. Close watchers of the social media site know that most of the time you only see around 6 percent of what your friends post. For organizations who want their followers to see their posts, it’s even less. But most users don’t know this is happening. As Alexis Madrigal explained, more than 60 percent of users in one study “had no idea that there even was a filtering algorithm, let alone one that looks at more than a thousand data signals to determine what to show a user.”

Worse, that filtering algorithm has increasingly turned into a pay-for-play system from news organizations. Want more people to see your content? Then “boost” your posts by shelling out some money. This already has turned Facebook into something of a two-tiered content sharing system, where the rich will inevitably see their stories go “viral” (if you can even call it that) much faster than will the poor. This inequality gap will only be exacerbated as more news organizations move over to publishing directly and the pressure—whether it be overt or implied—on those holding out increases.

Facebook, with its myriad acquisitions of potential competitors and promising upstarts— from Instagram to WhatsApp and many more—has itself well positioned to dominate Internet sharing for years to come. But with that dominance will come scrutiny. Along with increased wariness from news organizations, like Microsoft and Google before it, Facebook could one day face regulatory restrictions as well.

How will its algorithms handle stories posted directly to Facebook that question Facebook’s monopoly status? How will it handle news organizations questioning its lobbying ties with the government? Google is already a DC lobbying powerhouse, and you can bet Facebook, which is already beefing up its operations, will not be far behind. What about Facebook’s privacy practices? If the Washington Post posted its PRISM story about collusion between tech companies and the NSA directly on Facebook – a story that Facebook disputed – would its algorithms subtly suppress it? Would a Facebook employee ever so gently suggest that maybe the Washington Post may want to think twice before publishing, given what’s at stake for their relationship?

In the future, it may not be the New York Times that the White House pressures to stop publication of a particular story. They may also head straight for who actually controls whether millions will actually see the next explosive national security investigation: Facebook.

I don’t post these questions to accuse Facebook of doing any of these things currently. I have no evidence of that. But with so much control over content, and with so much to lose, such power is inevitably abused. Just ask the NSA. It’s the exact reason many worry of “turn-key tyranny” inside the US spy agency, even if—as some claim—the NSA is not currently abusing its massive domestic spying apparatus at all.

News organizations have always been at risk of bending to the will of their advertisers – and history is replete of examples of them doing just that. But the changing dynamics around Facebook are of a different order. Standard Oil or Pfizer or General Motors never had the power to ensure millions of New York Times subscribers would not get their paper the next day. Yet with one click, Facebook could pull off the modern-day equivalent.

Will this make it even less likely that a newspaper dares question its ultimate traffic keeper? After all, newspapers certainly need Facebook more than Facebook needs newspapers. Amazon CEO Jeff Bezos bought the Washington Post for $250 million dollars two years ago, breathing new life into a paper that was struggling financially for years. Facebook, which bought WhatsApp for $19 billion soon after, could buy 76 Washington Posts for that amount.

No, Facebook will not destroy the independence of news organizations overnight. News organizations aren’t going to stop publishing articles on their own sites, along with myriad other social media networks that might direct more traffic their way. And no, Facebook is not necessarily out to crush the news industry with ill intent. Facebook’s intentions with this newest collaboration could very well be honorable. Many of these problems exist to a certain extent whether news organizations decide to publish directly to Facebook or not.

But as it amasses even more power that we willingly hand over, we shouldn’t be surprised if it turns against us.

Has America ever needed a media watchdog more than now? Help us by joining CJR today

Trevor Timm is the executive director of Freedom of the Press Foundation, a non-profit organization that supports and defends journalism dedicated to transparency and accountability. He is also a twice-weekly columnist for the Guardian, where he writes about privacy, national security, and the media.