James Clapper, the director of national intelligence, has issued a blistering condemnation of Edward Snowden, calling the surveillance disclosures published by the Guardian and other news outlets a “perfect storm” that would endanger American lives.

Testifying before a rare and unusually raucous public session of the Senate intelligence committee that saw yet another evolution in the Obama administration’s defense of bulk domestic phone records collection, Clapper called on “Snowden and his accomplices” to return the documents the former National Security Agency contractor took, in order to minimize what he called the “profound damage that his disclosures have caused and continued to cause”.

Snowden has repeatedly said he acted alone in assembling and leaking a vast trove of information on the scope of US surveillance efforts, a conclusion also reportedly reached by the NSA’s official investigation into the Snowden leaks.

Asked if the journalists who possess leaked surveillance information counted in Clapper's definition of an "accomplice", Clapper spokesman Shawn Turner clarified: "Director Clapper was referring to anyone who is assisting Edward Snowden to further threaten our national security through the unauthorized disclosure of stolen documents related to lawful foreign intelligence collection programs."

Turner declined to be more specific.

At the hearing, senator Ron Wyden, the Oregon Democrat whose questioning last March ended with Clapper lying to the panel about the deliberate collection of Americans’ data, pressed Clapper to give public answers on surveillance activities on American information “sent over the web or stored in the cloud” – references to NSA’s so-called “upstream” collection capabilities, which allow the agency to harvest data in transit. He also questioned Clapper on whether the NSA had conducted “warrantless searches” for “specific” Americans’ identifying information in its vast databases of foreigners’ internet content, an authority first reported by the Guardian.

“Can you tell us today whether any such searches have ever been conducted?” Wyden asked.

“Senator Wyden, I think, at a threat hearing, this would ... I would prefer not to discuss this and have this as a separate subject. There are very complex legal issues here, I just don’t think this is the appropriate time or place,” Clapper said.

Wyden extracted a promise from Clapper to issue a declassified answer in 30 days.

Backed by the leaders of several intelligence agencies – but not the NSA director, Keith Alexander, who was not present – Clapper claimed Snowden’s disclosures had left the intelligence community less able to detect terrorist activity. Those testifying were less definitive about any specific dangers to the US that might result from what Snowden did, more often describing it as an over-the-horizon concern.

“It certainly puts us at risk of missing something that we are trying to see, which could lead to [an attack],” said Matthew Olsen, the director of the National Counterterrorism Center.

“Terrorists and other allies have gone to school,” Clapper said, employing some of the most fervent language that the intelligence agencies have used publicly to describe Snowden’s disclosures. The impact “includes putting the lives of members, or assets of, the intelligence community at risk”, Clapper said.

The counter-intelligence capabilities of al-Qaida are “increasingly good and, unfortunately, I think they just have to pick up the paper or do a Google search on what’s been leaked”, said John Brennan, the CIA director.

Lt Gen Michael Flynn, the director of the Defense Intelligence Agency, said the “greatest cost” of the NSA disclosures was “unknown today, but [what] we will likely face is costs of human lives on tomorrow’s battlefield, some place we put our mil[itary] forces when we ask them to go into harm's way”.

This particular hearing, known as the Worldwide Threat briefing, was where Clapper did major damage to his reputation last March. Under questioning from Wyden, Clapper said the government did “not wittingly” collect data on millions of Americans, a lie for which he would apologize to the panel, though not before changing his story about what prompted it.

Clapper first said it was the “least untruthful” answer he could give publicly. Then he said he “simply didn’t think” of the correct claimed legal authority under which the mass data collection occurs.

President Obama has stuck by Clapper, a decision that foreshadowed his recent call to preserve the vast majority of the NSA’s bulk surveillance authorities while transferring its phone records database to the custody of an undefined private entity. But six legislators, led by congressman Darrell Issa of California, wrote to Obama on Monday in an effort to get Clapper fired.

“The continued role of James Clapper as director of national intelligence is incompatible with the goal of restoring trust in our security programs and ensuring the highest level of transparency,” they wrote, and were just as quickly rebuffed by the White House.

Clapper’s defenders have said that Wyden placed the director in an untenable position by publicly querying him about a secret program, making his options either to lie or to decline to answer publicly, which they say would amount to public confirmation of a secret intelligence activity.

But on Tuesday, Clapper, joined by Brennan, opted to state instead that it was better to discuss certain unconfirmed intelligence activities in a classified hearing, where most of the panel’s work is conducted. The panel, usually a bastion of support for the intelligence agencies on Capitol Hill, featured sharp questioning and internal disagreement over surveillance and the CIA’s former torture programs.

Clapper reserved an answer on whether Russia had accessed the Snowden trove for a private session. Snowden has repeatedly stated that he did not take any documents to Russia, which granted him temporary asylum last summer after the US revoked his passport.

Clapper also declined to answer a question about European surveillance involving US businesses, and possibly upon them, in an open hearing.

Clapper did pledge to be more “transparent” in explaining surveillance actions in the future, in order to maintain public support for them. And while the NSA’s Alexander did not attend the hearing, he reportedly announced a former Homeland Security official, Rebecca Richards, as the surveillance agency’s first privacy and civil liberties officer.

Querying Clapper, Wyden said: “I don’t think this culture of misinformation is going to be easily fixed.”

Two powerful senators signalled their opposition to President Obama’s plans to move the NSA out of the business of bulk collection and storage of domestic call records.

Jay Rockefeller, the West Virginia Democrat and former intelligence committee chairman, said he “absolutely opposes contracting out this core government function”, saying the telecommunications firms would do a worse job of protecting US privacy and national security.

“I can’t tell you how strongly I feel about this. The president left us in a very interesting situation,” Rockefeller said.

His successor as chair, California Democrat Dianne Feinstein, who recently shepherded a bill through the committee that would entrench and expand NSA’s authority over bulk phone data, backed Rockefeller.

“In my view, he knows what he’s talking about.”

Angus King, the Maine independent, urged the intelligence chiefs to make a thorough case that the bulk collection of domestic phone data was necessary for preventing terrorism, rather than a useful but optional tool – as the latter, King said, was difficult to defend to constituents.

“It allows us to do in minutes what would otherwise take us hours” to determine if there was a domestic nexus to a foreign terrorist attack, said the FBI director, James Comey, who said “agility” was the primary value of bulk collection.

“Now, in most circumstances, the difference between hours and minutes isn’t going to be material, except when it matters most,” Comey said.

This is hardly the first time that the Obama administration’s defense of surveillance has changed, and it is an even further cry from the claims made by the NSA this summer that it had actually stopped looming terrorist attacks at home.

Members of the panel also feuded among themselves, as senators tried to out-duel each other to condemn Snowden, and varied in the extent to which they challenged Clapper.

Feinstein began the hearing by warning the committee to stick to “unclassified details”, a signal against pressing Clapper to address additional information about surveillance.

She ended it by permitting Wyden additional time to ask the intelligence chiefs to provide an example of when the spy agencies needed phone data urgently that was too old for telecoms to provide through normal processes.

“You had a long 10 seconds. Be grateful,” Feinstein said.