The US has admitted using a secret system to mine the systems of the biggest technology companies to spy on millions of people's online activity, overshadowing attempts by Barack Obama to force China to abandon its cyber-espionage program.

As concern mounted over the sweeping nature of US surveillance, the director of national intelligence, James Clapper, confirmed revelations by the Guardian that the National Security Agency uses companies such as Google, Facebook and Apple to obtain information that includes the content of emails and online files.

Coupled with the acknowledgement that authorities had undertaken a seven-year program to monitor the telephone calls of potentially millions of people in the US, it has become clear that the Obama administration has embraced and expanded the surveillance regime began under President Bush.

Clapper insisted that the internet surveillance program, known as Prism and disclosed by the Guardian and the Washington Post on Thursday, only covered communications with foreigners and did not target US citizens. "Information collected under this program is among the most important and valuable intelligence information we collect, and is used to protect our nation from a wide variety of threats," Clapper said.

He acknowledged that Section 702 of the Foreign Intelligence Surveillance Act was being used to "facilitate the acquisition of foreign intelligence information".

A secret 41-slide PowerPoint presentation obtained by the Guardian says that the information can be collected "directly from the servers" through the Prism system. The technology companies denied that direct access to servers was possible in this way, but they admitted complying with legal orders to turn over information.

Clapper attacked the disclosure as "reprehensible" for risking "important protections for the security of Americans".

More immediately, the admission places the US in an embarrassing position when it confronts Chinese leaders over their alleged use of cyber-espionage during a long-awaited summit in California on Friday.

Experts on US relations in Beijing said the revelations were bound to "weaken the US government's moral position" although they drew distinctions between the two approaches and expected the issue would still be raised. "Obviously the news breaking on the eve of the Sunnylands summit puts Obama in a much weaker position," added Linda Jakobson, east Asia program director at the Lowy Institute.

To push back against the growing scandal, Clapper also declassified aspects of a highly secretive acquisition of all Verizon's phone records first disclosed by the Guardian. Clapper took the extraordinary step late Thursday night to argue that the program operates "within the constraints of law" and "appropriately protect[s] privacy and civil liberties".

"The collection is broad in scope because more narrow collection would limit our ability to screen for and identify terrorism-related communications," Clapper said. Yet Clapper defended the broad, ongoing intelligence collection effort by saying that "only a small fraction" of the phone records – such as phone numbers and call – are ever scrutinized by intelligence analysts for connections to terrorism. Such scrutiny occurs according to "strict restrictions" overseen by the Justice Department and the special, secretive US surveillance court, he continued.

The document claims 'collection directly from the servers' of major US service providers.

Clapper reiterated that the content of phone calls is off-limits under the National Security Agency "metadata" collection program – while avoiding reference to the Prism system that sweeps up such content from nine participating internet companies. Clapper also repeatedly pointed out that some, but not all, members of Congress "have been fully and repeatedly briefed" on the program.

The secret document obtained by the Guardian shows that the Prism system facilitates extensive, in-depth surveillance on live communications and stored information. The NSA access was enabled by changes to US surveillance law introduced under President Bush and renewed under Obama in December 2012. The law allows for the targeting of any customers of participating firms who live outside the US, or those Americans whose communications include people outside the US.

The Guardian has verified the authenticity of the document, a 41-slide PowerPoint presentation – classified as top secret with no distribution to foreign allies – which was apparently used to train intelligence operatives on the capabilities of the program. The document claims "collection directly from the servers" of major US service providers.

Technology companies appeared not to be aware of how the NSA characterises the system. Apple said it had "never heard" of Prism. An Apple spokesman said: "We do not provide any government agency with direct access to our servers and any agency requesting customer data must get a court order."

A Google spokesman also said it did not provide officials with direct access. "Google cares deeply about the security of our users' data. We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government 'back door' into our systems, but Google does not have a 'back door' for the government to access private user data."

Legislators, particularly those serving on committees that oversee US intelligence, also confirmed the existence of the spy efforts, saying they have been in effect for at least six years – and jumped to their defence.

"These activities have led to the successful detection and disruption of at least one terrorist plot on American soil, possibly saving American lives," said the leadership of the House intelligence panel, Represenatives. Mike Rogers, a Republican, and Dutch Ruppersberger, a Democrat, in a joint statement.

But senator Ron Wyden, who for at least two years has warned about secret government interpretations of the Patriot Act authorising much larger surveillance efforts than the Obama administration has described, suggested the spying has not disrupted any such plots. "Based on several years of oversight, I believe that its value and effectiveness remain unclear," said Wyden, a Democratic member of the Senate intelligence committee.