Nearly 18 months after Edward Snowden’s disclosures upended the secret world of US surveillance, the US Senate has rejected the most politically viable effort to rein in the National Security Agency in almost four decades.

The USA Freedom Act, a bill introduced last year that sought to end the NSA’s ongoing daily collection of practically all US phone data, failed to reach a 60-vote threshold to cut off debate and move to passage.

Senators, mostly Republicans warning of leaving the country exposed to another terrorist attack, voted to beat back the bill, which had been warily backed by the Obama administration, technology giants and most civil libertarian groups.

It was the denouement to over a year’s worth of political drama, characterized by shifting alliances and a reduction in ambitions for constraining the NSA, even in a post-Snowden Congress.

Although the domestic phone data dragnet has not thwarted any terrorist attacks, in the lead-up to the vote critics savaged the bill as a gift to terrorists.

“God forbid we wake up tomorrow and Isil is in the United States,” said Florida Republican Marco Rubio, using an acronym for the Islamic State.

Afterwards, a downcast and impassioned Senator Patrick Leahy, the Vermont Democrat who sponsored the bill in the Senate, denounced “scare tactics” he said killed the bill.

‘Obviously I’m disappointed by tonight’s vote,’ Senator Leahy said after the vote, vowing ‘not to give up the fight’. Photograph: Win Mcnamee/Getty Images

“Obviously I’m disappointed by tonight’s vote,” Leahy said after the vote, vowing “not to give up the fight”.

The White House signaled its “strong support” for the bill on Monday. But that support came only after administration officials substantially weakened privacy protections and transparency additions in its House counterpart, all after a key House committee approved it.

Both the administration and the intelligence agencies fear that the defeat of the USA Freedom Act will result in the House declining to reauthorize broader domestic surveillance powers for the NSA and the FBI next year. The NSA and its allies accordingly opted to back the bill, despite having publicly expressed their reservations.

The failure of the Senate vote will pivot the surveillance debate in the next, Republican-dominated Congress to the question of re-authorizing Section 215 of the Patriot Act. That fight is likely to be among the chief priorities of the NSA and the FBI next year, backed by the White House.

In October, Leahy’s House partner on the USA Freedom Act, Wisconsin Republican James Sensenbrenner, predicted that the House of Representatives will not renew the expiring provision.

Another open question is whether the Obama administration will seek from the secret Fisa Court another 90-day extension of the bulk domestic phone records dragnet when its existing authorization expires on 11 December.

Privacy advocates, once enthusiastic about the USA Freedom Act, held their noses through its transformation into a consensus bill. While most considered the end of the NSA’s bulk domestic phone data dragnet significant enough to merit support, they lamented that the bill voted on Tuesday night no longer prevented the NSA or FBI from warrantlessly sifting through international communications databases for Americans’ identifying information, and that a public advocate created on the Fisa Court would have been insufficiently empowered.

Several feared that the legislation would not go far enough towards ending mass surveillance, given the NSA’s continued ability under the bill to obtain thousands of “call records” from telecommunications companies based on a single judicial order requiring merely “reasonable articulable suspicion” of connection to terrorism. No other NSA authorities exposed by Snowden, from the agency undermining encryption to any of its foreign mass-surveillance dragnets, would have been changed by the bill.

Other civil libertarians and NSA whistleblowers – including mass-surveillance insiders Thomas Drake, Bill Binney, Ed Loomis and Kirk Wiebe – opposed the USA Freedom Act, fearing that its vague legislative wording about what constitutes a “connection” between call records could inadvertently expand NSA authorities. The NSA has a history of expansive interpretation of legal provisions meant to restrict it, including the Patriot Act measures it claims authorize its domestic phone data dragnets.

Last week, Senator Rand Paul, the Kentucky civil libertarian and potential Republican presidential contender, said the USA Freedom Act was too weak for him to support. He is instead urging the expiration of certain surveillance authorities under the Patriot Act next year, as he did unsuccessfully in 2011. He voted against cloture on Tuesday night.