National Security Agency (NSA) Director Michael Rogers testifies before a House (Select) Intelligence Committee hearing on "Cybersecurity Threats: The Way Forward" on Capitol Hill in Washington November 20, 2014. Joshua Rodgers/Reuters

NEW YORK (Reuters) - A federal appeals court on Thursday revived a challenge to a controversial National Security Agency program that collected the records of millions of Americans' phone calls, saying the program was not authorized by Congress.

The 2nd U.S. Circuit Court of Appeals in Manhattan said a lower court judge erred in dismissing a lawsuit by the American Civil Liberties Union that challenged the constitutionality of the surveillance on the ground it violated people's privacy.

At issue was the NSA's collection of "bulk telephony metadata," a program whose existence was first disclosed by former NSA contractor Edward Snowden.

In December 2013, U.S. District Judge William Pauley in Manhattan dismissed the ACLU lawsuit, saying the NSA program was a "counter-punch" by the government to aid its efforts to fight terrorism.

Writing for a three-judge appeals court panel, however, Circuit Judge Gerard Lynch said Congressdid not authorize the NSA program under a section of the Patriot Act governing how investigators may collect information to fight terrorism.

"Such expansive development of government repositories of formerly private records would be an unprecedented contraction of the privacy expectations of all Americans," Lynch wrote in a 97-page decision.

"Perhaps such a contraction is required by national security needs in the face of the dangers of contemporary domestic and international terrorism," he added. "But we would expect such a momentous decision to be preceded by substantial debate, and expressed in unmistakable language. There is no evidence of such a debate."

Thursday's decision did not resolve the issue of whether the NSA program violated the bar against warrantless searches under the Fourth Amendment. The 2nd Circuit returned the case to Pauley for further proceedings. It upheld his denial of a preliminary injunction to stop the collection of records.

The U.S. Department of Justice did not immediately respond to a request for comment. The ACLU did not immediately respond to a similar request.

The case is American Civil Liberties Union et al v. Clapper et al, 2nd U.S. Circuit Court of Appeals, No. 14-42.

(Reporting by Jonathan Stempel in New York; Editing by Chizu Nomiyama and Frances Kerry)