“The intelligence community may have finally overextended itself. As well as being out of government control, it may have also expanded beyond its own control.” Truer words could hardly be written about the fiascos facing the U.S. National Security Agency today because of its massive global eavesdropping, and the Central Intelligence Agency, because of its all-too-lethal drones.

At a meeting in Brussels Thursday, European leaders vied with each other to express their outrage at the NSA program, which reportedly gathered information on tens of millions (actually, if you extrapolate the numbers, hundreds of millions) of phone calls on the Continent, and also, by the by, appears to have tuned in on German Chancellor Angela Merkel’s official cell phone.

Meanwhile Pakistani Prime Minister Nawaz Sharif made it clear in a public statement while visiting the White House: no more drones! Never mind that his predecessors reportedly signed off on the not-so-secret CIA drone program blowing away jihadist bad guys and, all too often, bystanders in the remote mountains near Afghanistan.

Even the intelligence operations of the redoubtable New York City Police Department are under fire. A coalition of 125 civil rights, religious and community groups has just written to the U.S. Department of Justice to demand a federal investigation of the surveillance activities the cops conduct in New York’s Muslim communities.

The Obama administration is blaming leakers like Edward Snowden, the former private contractor for the NSA who defected (yes, let’s use that word) to Russia, for some of these problems. But the White House is sounding more and more defensive and ineffectual. After Merkel called President Barack Obama to give him a piece of her mind, his spokesman Jay Carney told the press that “the President assured the Chancellor that the United States is not monitoring and will not monitor the communications of the Chancellor.” Carney refused to be drawn out on the question of whether the United States used to eavesdrop on Merkel’s calls.

This morning, The Guardian, which has been the vehicle for many of Snowden’s leaks, reported that back in 2006, under the George W. Bush administration, the NSA was asking the “customers” for its intelligence products in the White House, the State Department, and elsewhere to share the private numbers of their high level contacts overseas.

The frontpage story is based on a single memo that reported, “In one recent case, a US official provided NSA with 200 phone numbers to 35 world leaders.” In truth, this wasn’t very useful: 157 of those numbers were available to the public already. The 43 remaining and some others were “tasked,” according to the memo, but produced “little reportable intelligence.”

Whew. Bugging our closest allies’ leaders for next to nothing. “Overextended” is the word. But, did I mention that the line about that problem of overreach quoted at this beginning of this column was written almost 30 years ago? It comes from Philip Knightley’s classic study, The Second Oldest Profession, about spies and spying in the last century.

In fact there is something terribly cyclical—and dangerous—about the ebb and flow of intelligence community abuses and public reactions over the years. The spooks overreach, the public overreacts, and what should be the reasonable business of spying gets vilified and cut back until, one day, disaster strikes, and the public wonders why “nobody warned us.”

Yes, think 9/11.

Steward A. Baker, a former general counsel at the NSA and assistant secretary at the Department of Homeland Security when it was formed after the attacks on the United States, likens the current flap to a bleak version of the film Groundhog Day. The same story is lived again and again. He blames the pressure from civil libertarians in the late 1990s for the compartmentalization of intelligence—the court-ordered “wall” between the Federal Bureau of Investigation and the CIA—that let two known al Qaeda operatives hang out in plain sight in Southern California until they blew themselves up in the atrocity of September 11, 2001.

“I never want to live through that particular Groundhog Day again,” Baker told the Senate Judiciary Committee last July. “That’s why I’m here. I am afraid that hyped and distorted press reports orchestrated by Edward Snowden and his allies may cause us—or other nations—to construct new restraints on our intelligence gathering, restraints that will leave us vulnerable to another security disaster.”

Of course, intelligence agencies never blame themselves for their failure. As Knightley pointed out way back when, over the years they “have brainwashed successive governments into accepting three propositions that ensure their survival and expansion”: first, that “in the secret world it may be impossible to distinguish success from failure”; second, “that failure can be due to incorrect analysis of the agency’s accurate information—the warning was there but the government failed to heed it”; and third, “that the agency could have offered timely warning had it not been starved of funds.”

These were precisely the kinds of arguments made after the failure to predict or to stop 9/11, and, much more recently, after the complete failure to predict what used to be known as the Arab Spring.

But the technical changes since Knightley was writing have increased all these problems exponentially. In the 1980s the intelligence community in the United States already had difficulties with the sheer volume of stuff it collected. “It now produces so much information, such an all-sources glut of words, images and electronic data that the number of intelligence officers who can understand it all, who see the overall pictures, is rapidly declining,” Knightley wrote.

My favorite example of this was in Iraq during and just after the American invasion in 2003, when looters were ransacking Baghdad like the Forty Thieves in The Arabian Nights. U.S. intelligence agencies were monitoring phone calls all over the city and kept hearing the name “Ali Baba,” which loomed large on their relational diagrams and power points. It took a while for them to realize Iraqis used that word to describe all the thieves.

As James Bamford pointed out back in 2009 in great detail in his book The Shadow Factory, when global communications moved from wires and satellites to undersea and underground fiber optic cables carrying millions of calls and emails at a time, the whole business of intelligence gathering shifted its emphasis to what’s called “collection first.”

It was no longer practical or indeed possible to put alligator clips on the landline of a terrorist. He was using the Internet for his emails and his phone calls. So, virtually all data moving through the fiber optic lines that crisscross the globe has to be sucked into the NSA’s computers, then sophisticated filters (many of which were developed by the Israelis, as Bamford points out) are used to sift the torrent of communications until only a very narrow range of suspect ones actually are monitored. It’s a little like fishing in Niagara Falls, trying to catch two or three fish at a time.

How do you figure out what phones or email accounts to target? You look at who’s communicating with whom, and that information is to be found in the metadata that the NSA has collected from the major Internet service providers and other communications companies.

As Baker notes, this may be shocking to some, but it’s perfectly legal in the United States, since “the Supreme Court has held that such records are not protected by the Fourth Amendment, since they’ve already been given to a third party,” the private company, which then shared that information with the U.S. government. “Google is the real Big Brother,” as people in the intelligence community like to say.

In the meantime, of course, other countries are doing their best to do just what the NSA does, albeit with fewer resources and inferior technology. Britain and France both have vast electronic surveillance programs, Germany’s is extensive as well, and despite public posturing by the politicians, the level of public cynicism in Europe is high. After a Snowden revelation that 1.8 million Dutch phone calls had been “monitored” in a month (meaning metadata was collected and perhaps a handful actually were read or listened to) a conservative member of the Dutch parliament said blandly, “It sounds like somewhat more than the number we are watching ourselves through our own services.”

In Germany, much of the reaction to Merkel’s indignation with Obama has been indignation with Merkel for not making much of a fuss about Snowden’s revelations until she discovered her own phone was an NSA target. “She clearly doesn’t care about the fundamental rights of the people,” writes Ludwig Greven in Die Zeit.

Virtually forgotten in all this uproar: the Russians, under President (and former KGB agent) Vladimir Putin, continue to conduct vast spying operations around the world. So do the Chinese. Indeed, according to Baker, Merkel’s computer network was hacked by China a few years ago. There’s no record of her calling Beijing to complain.

So, yes, the Americans overreached. They need to be reined in. There should be better understandings with the allies. But let’s not kid ourselves. The world’s second oldest profession, spying, is here to stay.

—With Nadette De Visser in Amsterdam