If the government stops collecting our phone records, is our privacy protected?

That’s a common assumption in many debates about the National Security Agency. We’ve come to think of privacy as a binary question, with government as the sole threat. Now we have to think about other threats, because President Obama is proposing to outsource the NSA’s phone records program.

In his speech on Friday about reforming the NSA, Obama explained that the phone program is divided into two stages. The first stage is collection of the records, which show the date, time, and duration of each call from one number to another. In the second stage, the database of records is searched (“queried”) for numbers that have interacted, directly or indirectly, with numbers linked to suspected terrorists. In theory, the two stages can be administered and regulated separately.

In fact, the collection process begins in the private sector. What the NSA has done, Obama noted, is simply “a consolidation of phone records that the companies already retained for business purposes.” The panel of experts assigned by Obama to investigate the NSA (the Review Group on Intelligence and Communications Technologies) “turned up no indication that this database has been intentionally abused,” said Obama. Nevertheless, he acknowledged, “without proper safeguards, this type of program could be used to yield more information about our private lives.”

So Obama offered more safeguards. He promised to replace the program with a system capable of doing the same things, but “without the government holding this metadata itself.” The telecommunications companies, for instance, might be enlisted to retain their records, “with government accessing information as needed.” Or a “third party” might keep the records.

Yesterday on Face the Nation, Bob Schieffer second-guessed this idea. Alluding to recent security breaches at retailers, he asked Rep. Mike Rogers, the chairman of the House intelligence committee: “Do you think the private sector can do this better than the government? I mean, I look at what happened here at Target, what’s happened at Neiman Marcus.”

Rogers agreed. Within the government, he argued, there were several layers of review over use of the phone data:

If you move all that to the private sector, you lose all of the review. That goes away. And you open it up to privacy concerns I don’t think we talked about. Divorce lawyers are going to have a heyday. Private detectives on any civil matter anywhere in the country are going to have a heyday.

Schieffer then turned to Sen. Mark Udall of Colorado, an NSA critic. And here’s where the conversation took a disquieting turn. Udall dismissed Rogers’ warning as a “parade of horribles.” “The phone companies,” Udall assured viewers, created these records only because “it’s their business model to collect this data. They’re not going to use that data in ways that will break faith with their customers.”

Wait a minute. Here’s the guy who has been brought onto the show to represent civil liberties. As long as the threat in question is the NSA, he does the job. He doesn’t accept the government’s claims of good will. He’s not impressed by the absence of known deliberate abuse. He wants clear safeguards and independent oversight.

But when the conversation shifts from the government to the telecom providers, Udall drops his guard. Suddenly the records collection is just business, and the collectors can be trusted not to “break faith with their customers.”

A similar thing happened on Meet the Press. Sen. Dianne Feinstein, the chair of the Senate intelligence committee, noted that the NSA has rules about who can access its phone records database: 22 people who, at least in theory, are vetted and supervised. If the data retention is outsourced, she asked, “How do you provide that level of supervision?” David Gregory pushed the point further, arguing that Amazon, Google, and other companies are “not only compiling data, but sharing that data, selling that data.” He asked Reddit co-founder Alexis Ohanian: “Is that just as much of a concern as what the government is doing?”

Like Udall, Ohanian was supposed to be the civil liberties advocate. And, like Udall, he deflected the question. As to expectations of privacy, he assured viewers, “That’s a contract that we have with our service providers.” Promises from the government weren’t to be counted on, but apparently, promises from Internet companies were. Indeed, Ohanian blamed the NSA for the private sector’s loss of public confidence: “Countries and citizens around the world no longer want to do business with American companies, because they no longer trust that their private data is safe. I’m an entrepreneur. I’m an investor.”

No matter what you think of the NSA, this selective skepticism makes no sense. After all the security breaches, privacy issues, and consumer data practices that have been exposed at one company after another—Target, Neiman Marcus, Facebook, Yahoo, Adobe, Google, Snapchat, Ford, and others—you’d think that civil libertarians would worry about private-sector data management, not just about the government. Would delegating the retention of phone records to telecom providers or a “third party” make the risks to privacy go away? Could it create new risks? Those are good questions.

In the heat of political battle, it’s easy to lock in on your adversary—in this case, defenders of the NSA—and dismiss their arguments. But never forget the principles that brought you into the fight. They’re bigger than the villain of the moment.