The spying scandal ripping Germany and the U.S. apart has deep roots. For decades, current and former U.S. intelligence officials say, the CIA has sent agents into Germany without Berlin’s knowledge—in part, out of concerns that the German security state was penetrated by its Russian neighbors.

The CIA and their German counterparts have worked closely for years on countering terrorists and thwarting Iran’s nuclear ambitions. But despite the cooperation on these issues, the CIA has also long held the view that Germany’s intelligence and security services have been penetrated by foreign agents.

Current and former U.S. intelligence officials who worked on the Germany file tell The Daily Beast that much of the CIA’s activities inside the country were not directed at prying secrets from the German state—at least ordinarily. Instead, they were focused primarily on Russian intelligence activities, foreign terrorist groups and Iranian technology procurement.

“We have had very serious problems with their counter-intelligence over the years,” one senior U.S. intelligence official told The Daily Beast. “In counter-terrorism and in other areas they some times have problems that are bigger than they acknowledge to themselves.”

This official declined to discuss the details of the current espionage case that is roiling the German government today.

According to German press accounts, the latest scandal began when a 31 year old man working for Germany’s intelligence service, the Bundesnachrichtendienst (BND) volunteered to spy for the United States All told he gave his CIA case officer, according to these accounts, 200 documents. For that he was paid $34,000.

But in an even stranger twist, it turns out the German spy was also working for the Russians. Current and former U.S. intelligence officials say this detail was important. “The BND was not a very good service for many years,” said one retired senior CIA case officer. “The station chiefs were always told to try to find out what they knew about us, but it was never much. We always worried that they had moles working for the Soviets and later the Russians.”

The German government publicly ordered the CIA’s station chief in Berlin to leave the country Thursday. A similar request was not issued to his Russian counterpart.

Russia’s connections to Germany extend far beyond its intelligence and security services, however. German defense contractors have helped train and equip the Russian military.

And, as Foreign Policy magazine recently noted, “Germany maintains closer relations to Russia than any other country in the European Union; Germany is Russia's third-largest trade partner… while Russia sells Germany over a third of its natural gas and oil.”

After leaving his post as Germany’s Chancellor in 2005, Gerhard Schroder became the head of the shareholder’s committee of the Nord Stream natural gas pipeline project that pumped Russia’s natural gas directly into Germany. The post drew harsh criticism from his political opposition because Schroder as Chancellor offered to cover one billion euros of the cost of constructing the pipeline. Schroder as a private citizen has said favorable things about Russia, most recently comparing Russia’s annexation of Crimea to the NATO intervention in Kosovo.

And of course, the forerunners to Russia’s modern spy services had plenty of experience operating on German soil. Vladimir Putin famously ran agents for the KGB from 1985 to 1990 out of Dresden, which was then in communist East Germany. His successors are still in the country, albeit on less friendly terms. "There is a huge Russian presence in Germany," said the senior U.S. intelligence official.

Part of the current concern about Russia’s activities in Germany stems from unease about Berlin’s equivalent of the FBI, known as the Bundesamt für Verfassungsschutz (BFV). One former U.S. intelligence officer who worked on European issues said the BFV had a strong reputation for identifying and neutralizing domestic threats inside Germany, but was not very good hunting so-called “moles” – foreign agents burrowed into their spy services. “I can tell you they never watched us very carefully at all,” this official said. “That is almost definitely going to change now.”

The CIA has conducted espionage operations inside Germany for decades without telling their counterparts. Of course, Germany is an ally. But there is no special understanding between the CIA and the BND, where the U.S. pledges not to recruit German agents and to inform the Germans about all of its activities inside their country. That kind of relationship does exist between the CIA and four other English speaking nations known as the “five eyes.” Along with the United States, the five eyes include Australia, Canada, New Zealand and the United Kingdom.

The retired CIA officer said the CIA’s relationship with the BND was similar to the agency’s relationship with the Israeli service, the Mossad. “We work with them, but we never fully trusted them,” he said.

On Friday, Bloomberg News reported that President Obama this week instructed the U.S. ambassador in Berlin to offer the German government a proposal that would upgrade the intelligence relationship to Germany to something similar to the five eyes agreement. But Germany’s Chancellor, Angela Merkel, already angered by last year’s revelation that the NSA spied on her personal cell phone, rejected the offer.

At the end of the day, many observers still think Germany and the United States will be able to salvage at least elements of their intelligence relationship. After all, the BND and the CIA share information on Iran's nuclear program, cooperate on counter-terrorism investigations, and even share the take of foreign agents—although not always with great results. The BND first provided the CIA in 2001 with information from the infamous Iraqi defector the CIA code-named "Curve Ball," who falsely claimed Saddam possessed mobile biological weapons labs.

German companies in the 1980s and 1990s were fleeced of sensitive nuclear technology and equipment by rogue proliferators like A.Q. Khan, the father of Pakistan's A-Bomb. The Americans and the Israelis have worked to sabotage German gear that Iran has tried to purchase on the black market.

The BND, however, has more restrictions on engaging in these kinds of dirty tricks. "They operate under more constraints than the US intelligence agencies in terms of what they can do," said David Albright, the president of the Institute for Science and International Security, a think tank that focuses on Iranian proliferation. He has worked with the German government on Iran’s nuclear issues. "This [latest] scandal could cause the political establishment to establish rules of cooperation that could lead to even less cooperation on Iran."

Albright said that despite these restrictions, the Germans have been anxious to cooperate more with the CIA. “On Iran and North Korea, their position is that they wanted to work more together. They have been disappointed the United States has not been willing to cooperate more."

“But,” he added, "what can happen is that these scandals could lead to a backlash in the German political decision making apparatus that will make it harder to cooperate with American intelligence.”