Stephen Colbert started his monologue one night this week by ridiculing an arcane yet significant move by Congress: Lawmakers voted to roll back a set of privacy rules that would prohibit internet providers from selling or sharing web browsing history without a consumer’s consent.

“I guarantee you, there is not one person, not one voter of any political stripe anywhere in America who asked for this,” Colbert said. “No one, no one in America stood up at a town hall [and] said, ‘Sir, I demand you let somebody else make money off my shameful desires!'”

President Donald Trump is not only expected to sign the legislation, his White House next wants to take aim at an even bigger set of rules of the road for the internet, net neutrality.

Colbert’s riffs, Democratic attacks, and a social media backlash may not be enough to stop the rollback of the privacy rules, but it may be just a smidgen of what the White House, Republican lawmakers, and a GOP-dominated FCC will face if and when they move to scuttle the current net neutrality protections.

The Republican majority across the federal government is on a mission to deregulate, eliminating what they see as the overreach of the Obama years, and arguing that the price paid has been chilled investment and fewer jobs.

But the anti-regulatory zeal can’t match the emotion that consumers have when it comes to keeping their access to the internet free and clear, and their own information protected and private.

Telecom and cable companies who provide internet service are on the defensive. On Friday, AT&T, Comcast, and USTelecom each issued statements to try to set the record straight.

Comcast’s Gerard Lewis wrote in a blog post that they will “revise our privacy policy to make more clear and prominent that, contrary to the many inaccurate statements and reports, we do not sell our customers’ individual web browsing information to third parties and that we do not share sensitive information unless our customers have affirmatively opted in to allow that to occur.”

Still, he said that “if a customer does not want us to use other, non-sensitive data to send them targeted ads, we offer them the ability to opt out of receiving such targeted ads.”

The FCC passed the privacy rules just last year, when it still had a Democratic majority. Broadband providers opposed them from the start, with the main argument that the rules were more onerous for them than for the likes of internet sites like Google and Facebook, under the jurisdiction of the Federal Trade Commission. White House spokesman Sean Spicer told reporters on Thursday that the repeal would lead to an “equal playing field.”

Jonathan Spalter, president and CEO of the trade group USTelecom, argued that internet service providers were being singled out for heavier regulations than internet sites, which do share browsing history.

“Consumers’ browsing history is bought and sold across massive online advertising networks every day,” he wrote.

Yet that still doesn’t change the fact that what Congress has done is block the FCC from imposing rules that would prevent ISPs from sharing or selling that browsing information without consumers’ consent.

“You don’t lower the bar to zero; you raise the bar for everyone else,” says Gigi Sohn, a longtime public interest advocate who was counselor to the previous FCC chairman, Tom Wheeler.

The average consumer, she says, doesn’t care whether the rules come from the FCC or the FTC — they just want assurances that their information, including browsing history, is protected.

Public interest groups like Fight for the Future are planning to place billboards in districts of lawmakers who pledged to roll back the rules, and some groups are starting protest campaigns to buy the sensitive data of members of Congress, however doubtful that will be.

The point is that when it comes to reversing the current net neutrality rules, the White House, the Republican Congress, and the GOP’s FCC majority may have the means to do so, but they will be hard pressed to match the other side when it comes to their message and mobilization.

The FCC reclassified internet service as a common carrier and passed the net neutrality rules in 2015 after it was inundated with almost 4 million comments, the majority to impose robust regulations. Public interest groups demonstrated outside the agency’s D.C. offices, and even blocked Wheeler from leaving his home driveway, before he decided which approach to take on net neutrality.

Congress’ repeal of the privacy rules moved quickly through the Senate and House while much of the media focus was on healthcare reform, Neil Gorsuch, and Russia.

Reversing net neutrality, Sohn notes, is likely to take months, particularly if FCC chairman Ajit Pai moves to do so within the FCC. Public interest groups are readying for a fight, and, like they have in previous battles, are likely to characterize it as an issue where internet freedom is at stake.

“This is something that is popular, that people have had for two years, and taking it away is going to have a lot of blowback,” Sohn says.

Internet providers are pushing for a reversal of the rules, with the rationale being that they stifle investment, but they will have to contend with a PR issue. They are not exactly the most popular companies out there. On late-night TV, they are often the butt of jokes.

“I can’t believe they’re publicly taking the side of big internet cable companies,” Colbert said of lawmakers who voted for the legislation to roll back the privacy rules. “Taking the side of a cable company? The only thing less popular would be if they passed a bill allowing traffic jams to call you during dinner, to give you gonorrhea.”