US National Security Agency (NSA) whistleblower Edward Snowden speaks to European officials via videoconference during a parliamentary hearing on improving the protection of whistleblowers, at the Council of Europe in Strasbourg, eastern France, on June 24, 2014. AFP PHOTO/FREDERICK FLORIN (Photo credit should read FREDERICK FLORIN/AFP/Getty Images)

NEW YORK -- Judges considering the fate of the National Security Agency's call tracking program in an appeals hearing on Tuesday repeatedly expressed concerns over its vast reach.

The government is essentially making "an argument that says nobody's got any constitutional privacy rights in anything anymore" if they hand over information to third parties, said Judge Gerard Lynch.

But the three-judge panel from the U.S. Court of Appeals for the 2nd Circuit also seemed worried about acting too quickly given ongoing reform efforts in Congress and the likelihood that the Supreme Court will have the final say on the controversial surveillance program.

The panel was hearing an appeal in a lawsuit the American Civil Liberties Union first brought against the government days after Edward Snowden's leaks were published. U.S. District Judge William Pauley ruled against the ACLU in that case in December, finding the program lawful and constitutional.

A judge in Washington, D.C., ruled in a separate lawsuit in December that the call program was likely unconstitutional. The parallel lawsuits proceeding in two federal circuits sets up the possibility that the Supreme Court will be forced to step in to decide the question.

The appeals panel heard arguments from the ACLU and a Justice Department lawyer for more than an hour and a half in a proceeding that was, unusually, broadcast on C-SPAN.

"We've given it probably more time than you'll get in the Supreme Court," Lynch joked.

The ACLU argues in the case that the call records program fails on both constitutional and statutory grounds. The 2001 Patriot Act authorized the government to collect business records if they were deemed "relevant' to a terrorism investigation. The civil liberties group says that calling every American's phone records "relevant" to terrorism is a twisted interpretation of the law.

"If everything is relevant, the relevance simply drops out. It doesn't exist," Judge Robert Sack said Tuesday, seeming to agree. "The notion that everything is relevant is very troubling, at least."

But Assistant Attorney General Stuart Delery argued that Congress had specifically precluded ordinary federal courts from considering the meaning of that section of the Patriot Act, despite concerns that have been raised that not all legislators were thoroughly briefed on the phone program.

The government must collect almost every American's phone records, argued Delery, because "going in, the government doesn't know which of the metadata might reveal an important connection to an identified terrorist."

The panel gave no indication as to when it might decide the appeal in the case.

Congress is also wrestling with how to contain the public outrage generated since the Snowden revelations began in June 2013. The House passed a gutted version of NSA reform in May. Senate Judiciary Chairman Patrick Leahy (D-Vt.) is pushing a tougher measure in his chamber -- but it's unclear whether any changes will pass into law in the divided Congress.

"You'd be unwise to expect Congress, this Congress or the next, to act so soon," said ACLU lawyer Alexander Abdo.

In the meantime, Abdo said, Americans' privacy is being violated on a vast scale.