I t began as routinely as any other passenger flight. At gate 15 of New York City’s JFK Airport, more than 200 men, women, and children stood in line as they waited to board a Boeing 747. They were on their way to Seoul, South Korea’s capital city. But none would ever make it to their destination. About 14 hours after its departure, the plane was cruising at around 35,000 feet not far from the north of Japan when it was shot out of the sky. The downing of Korean Airlines Flight 007 occurred on September 1, 1983, in what was one of the Cold War’s most shocking incidents. The plane had veered off course and for a short time entered Soviet airspace. At Dolinsk-Sokol military base, Soviet commanders dispatched two fighter jets and issued an order to “destroy the intruder.” The plane was hit once by an air-to-air missile and plummeted into the sea, killing all passengers and crew. President Ronald Reagan declared it a “crime against humanity,” marking the dawn of a volatile new chapter in relations between the United States and the Soviet Union. Soon, tensions would escalate to a level not seen since the Cuban missile crisis, which 20 years earlier had brought the world to the brink of nuclear war.

Photo: Kim Chon-Kil/AP

As the international confrontation between the two adversaries played out publicly, behind closed doors another problem — which has never before been revealed — was developing. The U.S. and one of its closest allies, Japan, were embroiled in a dispute involving secret surveillance. Soviet officials were flat-out denying they had any role in shooting down the jet. At a spy base on Japanese territory, however, communications had been intercepted proving the Soviet military was the perpetrator. The U.S. wanted to obtain copies of the tapes but had to first receive approval from the head of a shadowy Japanese surveillance organization known as the “G2 Annex.” After some bureaucratic wrangling, the Japanese eventually signed off on the release and the highly sensitive recordings were sent to Washington. From there, the tapes were forwarded to New York City, where U.S. Ambassador Jeane Kirkpatrick brought them to the United Nations headquarters in Manhattan. On September 6 — just five days after the Korean Airlines jet was shot down — Kirkpatrick attended a meeting at the U.N. Security Council where she blasted the Soviet Union for telling “lies, half lies and excuses” about its involvement in the downing of the plane. She then proceeded to play the copy of the intercepted conversations, stating that the evidence was being presented in “cooperation with the government of Japan.” The case Kirkpatrick put forward against the Soviets was irrefutable and damning. But Japan’s spying capabilities had now been exposed — and the country’s officials were not pleased about it. The G2 Annex received new orders limiting its cooperation with the U.S., which affected the NSA’s relationship with its Japanese counterparts for the better part of a decade, at least until the Cold War ended in the early 1990s. The details about the Korean Airlines case are revealed in classified National Security Agency documents, obtained by The Intercept from the whistleblower Edward Snowden. The documents, published Monday in collaboration with Japanese news broadcaster NHK, reveal the complicated relationship the NSA has maintained with Japan over a period of more than six decades. Japan has allowed NSA to maintain at least three bases on its territory and contributed more than half a billion dollars to help finance the NSA’s facilities and operations. In return, NSA has kitted out Japanese spies with powerful surveillance tools and shared intelligence with them. However, there is a duplicitous dimension to the partnership. While the NSA has maintained friendly ties with its Japanese counterparts and benefited from their financial generosity, at the same time it has secretly spied on Japanese officials and institutions. The NSA declined to comment for this story.

View of the radioactive plume from the bomb dropped on Nagasaki, as seen from 9.6 kilometers away, in Koyagi-jima, Japan, on Aug. 9, 1945. Photo: Hiromichi Matsuda/Nagasaki Atomic Bomb Museum/Getty Images

O n August 14, 1945, Japan announced its unconditional surrender just days after U.S. Air Force planes dropped two atomic bombs on the cities of Nagasaki and Hiroshima, killing more than 100,000 people. The war was over, but as part of the peace agreement, Japan agreed to U.S. military occupation. American forces — led by Gen. Douglas MacArthur — drafted a new Japanese constitution and reformed the country’s parliamentary system. In April 1952, Japan’s sovereignty was restored, but the U.S. continued to maintain a major presence in the country — and that is where the NSA’s story begins. According to the agency’s documents, its relationship with Japan dates back to the 1950s. NSA’s presence in the country was for many years managed out of a “cover office” in the Minato area of downtown Tokyo, within a U.S. military compound called the Hardy Barracks. From there, NSA maintained close relations with a Japanese surveillance agency that it refers to as Japan’s Directorate for Signals Intelligence, or SIGINT. At first, the NSA appears to have kept a low profile in Japan, concealing details about its presence and operating undercover. But as its relationship with the country developed, that changed. By 2007, the agency had determined that “cover operations are no longer required” and it relocated its main office in Japan to a space within the U.S. Embassy in Tokyo. “NSA’s partnership with Japan continues to grow in importance,” the agency noted in a classified October 2007 report, adding that it planned to take the country “to the next level as an intelligence partner with the U.S.” Beyond Tokyo, NSA has a presence today at several other facilities in Japan. The most important of these is located at a large U.S. airbase in Misawa, about 400 miles north of Tokyo. At what it calls its “Misawa Security Operations Center,” the agency carries out a mission under the code name LADYLOVE. Using about a dozen powerful antennas contained within large golf ball-like white domes, it vacuums up communications — including phone calls, faxes, and internet data — that are transmitted across satellites in the Asia-Pacific region.

Photo: Haraz N. Ghanbari/AP

As of March 2009, Misawa was being used to monitor “over 8,000 signals on 16 targeted satellites,” one NSA document noted. At the same time, the agency was working on beefing up the spy hub’s systems, so that it could meet a challenge set by then-Director Keith Alexander to “collect it all” — meaning, to sweep up as many communications as possible. Misawa’s NSA employees responded to Alexander’s call by developing technology to automatically scan and process more satellite signals. “There are multitude of possibilities,” one Misawa-based NSA engineer reported, predicting that the base would soon be “one step closer to ‘collecting it all.’” Strategically, Japan is one of the NSA’s most valuable partners. Because of its close proximity to major U.S. rivals like China and Russia, it has been used as a launching pad to spy on those countries. But NSA’s operations in Japan are not limited to monitoring the communications of nearby adversaries. At Misawa, the NSA deployed programs called APPARITION and GHOSTHUNTER, which pinpoint the locations of people accessing the internet across the Middle East and North Africa. NSA documents detailing GHOSTHUNTER’s deployment at the NSA’s British base Menwith Hill state the program was used to facilitate lethal strikes, enabling “a significant number of capture-kill operations” against alleged terrorists. One November 2008 document noted that Misawa had proved particularly useful in tracking down terror suspects in Afghanistan and Pakistan, and was also being used in an effort to identify targets in Indonesia. Over the past decade, the NSA’s tactics have evolved dramatically — and it has rolled out new and more controversial methods. By 2010, with the internet surging in popularity, the agency was continuing to focus on long-established spying tactics like eavesdropping on phone calls, but it was increasingly adopting more aggressive methods, such as hacking into its targets’ computers. At Misawa, the NSA began integrating hacking operations into its repertoire of capabilities. One such method it deployed at the base is called a “Quantum Insert” attack, which involves monitoring the internet browsing habits of people targeted for surveillance, before covertly redirecting them to a malicious website or server that infects their computers with an “implant.” The implant then collects data from the infected computer and returns it to the NSA for analysis. “If we can get the target to visit us in some sort of web browser, we can probably own them,” an NSA employee claims in one document describing the hacking techniques. “The only limitation is the ‘how.’”

U.S. Marine Corps MV-22 Osprey aircraft sit on the tarmac at U.S. Marine Corps Air Station Futenma on Okinawa Island, Japan, on May 19, 2015. Photo: Hitoshi Maeshiro/EPA/Redux

T he Yokota Air Base, another U.S. military facility, sits at the foothills of Okutama mountains near the city of Fussa. The base is about a 90-minute drive west from central Tokyo and houses more than 3,400 personnel. According to the U.S. Air Force, Yokota’s function is to “enhance the U.S. deterrent posture and, if necessary, provide fighter and military airlift support for offensive air operations.” But it also serves another, more secret, purpose. NSA documents reveal that Yokota is home to what the agency calls its Engineering Support Facility, which supplies equipment used for surveillance operations across the world. In 2004, the agency opened a major new 32,000 square foot building at the site – about half the size of a football field – for the repair and manufacture of surveillance antennas it said would be used in places like Afghanistan, Korea, Thailand, the Balkans, Iraq, Central and South America, and Cyprus. The construction cost $6.6 million, which was paid almost entirely by the government of Japan, a July 2004 NSA report stated. Within the facility, Japan would finance the staff as well, the report noted, including seven designers, machinists, and other specialists, who were collectively receiving salaries worth $375,000. About 1,200 miles southwest of Yokota is the NSA’s most remote Japanese spying station, located on the island of Okinawa at a large U.S. Marine Corps base called Camp Hansen. It, too, has greatly benefited from a massive injection of Japanese money. In the early 2000s, NSA constructed a state-of-the-art surveillance facility on the island, paid for in full by Japan at a cost of some $500 million, according to the agency’s documents. The site was carved out of a “dense, hilly area” called Landing Zone Ostrich that the Marines had previously used for jungle training. The facility, built to include an “antenna field” for its spying missions, was designed to be low profile, blending in with the landscape. It replaced a previous spy hub NSA had maintained on Okinawa that the island’s Japanese residents had complained was unsightly. The role of the remote eavesdropping station is to collect high-frequency communications signals as part of a mission called STAKECLAIM. The NSA does not appear to have a large number of employees stationed on the island; instead, it remotely operates the Okinawa facility from a “24-hour collection operations center” in Hawaii. Hiroshi Miyashita, a former Japanese government data protection official, told The Intercept that Japan’s funding of U.S. intelligence activities is withheld from public disclosure under a state secrecy law, which he criticized. “It’s our money — Japanese taxpayers’ money,” he said. “We should know how much was spent for intelligence activities in Japan.” Miyashita, now an associate professor at Chuo University in Tokyo, said it was his understanding that NSA operates in the country outside Japan’s legal jurisdiction due to an agreement that grants U.S. military facilities in Japan extraterritoriality. “There is no oversight mechanism,” Miyashita said. “There is limited knowledge of activities within the bases.”

As recently as 2013, the NSA claimed to maintain “robust” working relations with its Japanese counterparts. The agency has two surveillance partners in Japan: the Directorate for SIGINT, and the Japanese National Police Agency. Japan has collaborated closely with the NSA on monitoring the communications of neighboring countries, and it also appears to rely heavily on U.S.-provided intelligence about North Korean missile launches. As of February 2013, the NSA was increasingly collaborating with its Japanese counterparts on cybersecurity issues. And in September 2012, Japan began sharing information with the NSA that could be used to identify particular kinds of malicious software being used by hackers. This was the first time the country had shared this kind of data and the NSA viewed it as highly valuable, potentially leading to the prevention or detection of hacking attacks on “critical U.S. corporate information systems.”

“Japanese citizens know almost nothing about Japanese government surveillance. It is extremely secret.”

In return, the NSA has provided Japanese spies with training, and it has also furnished them with some of its most powerful spying tools. An April 2013 document revealed that the NSA had provided the Japanese Directorate for SIGINT with an installation of XKEYSCORE, a mass surveillance system the NSA describes as its “widest reaching” for sweeping up data from computer networks, monitoring “nearly everything a typical user does on the internet.” Igeta Daisuke, a Japanese lawyer who specializes in civil liberties cases, said that the XKEYSCORE revelation was “very important” for the country. The Japanese government’s use of the system could violate Japan’s Constitution, which protects privacy rights, Daisuke told The Intercept. He added that Japan has a limited legal framework covering surveillance issues, largely because the scope of the government’s spying has never before been disclosed, debated, or ruled upon by judges. “Japanese citizens know almost nothing about Japanese government surveillance,” said Daisuke. “It is extremely secret.” The Japanese government’s defense ministry, which oversees the country’s surveillance capabilitites, declined to comment.

The Bank of Japan building in Tokyo on July 14, 2006. Photo: Kazuhiro Nogi/AFP/Getty Images

T he NSA works with a diverse range of counterparts in countries across the world — from the United Kingdom and Sweden to Saudi Arabia and Ethiopia. But the agency’s partnership with Japan is one of its most complex and seems tainted by a degree of distrust, highlighted by the dramatic aftermath of the Korean Airlines incident in 1983. In a November 2008 document, one of NSA’s then most senior officials in Japan offered an insight into the relationship. He described the Japanese as “very accomplished” at conducting signals intelligence but lamented that they were excessively secretive. The country’s spies were “still caught in a Cold War way of doing business,” the official wrote. “They treat SIGINT as a special-access program — the most sensitive program they have. The result is that they are rather stove-piped, somewhat like NSA was 10-or-more years ago.” The NSA participates in a group called the SIGINT Seniors Pacific, which has included surveillance agencies from Australia, Canada, the United Kingdom, France, India, New Zealand, Thailand, South Korea, and Singapore. The group keeps tabs on security issues in the Asia-Pacific region — issues of great interest to Japan, given its geographic location. Yet the country refused to join the meetings. “Japan was the only nation who was actually offered membership but turned it down,” wrote one NSA employee in a March 2007 document. “At the time, Japan expressed concerns that unintended disclosure of its participation would be too high a risk and had other reasons as well.” Some of the difficulties have directly impacted the NSA’s operations. According to the agency’s documents, for many years Japan participated in a surveillance program called CROSSHAIR, which involved sharing intelligence gathered from high-frequency signals. However, in 2009, the country abruptly ceased its participation in the program. Four years later, the issue was still causing NSA concern. Ahead of a February 2013 meeting the agency had scheduled with the deputy director of Japan’s Directorate for SIGINT, it prepared a briefing document that outlined the CROSSHAIR problem and warned of a “potential landmine” associated with the discussions. “In the past, the partner has mistakenly perceived that NSA was trying to force [the Directorate for SIGINT] to use U.S. technical solutions in place of their own,” the memo stated. “When this occurred, the partner reacted in a strong, negative manner.” But while NSA employees may walk on eggshells with Japan during face-to-face meetings, they have taken a different approach on a covert level. An NSA document from May 2006 indicated that a division of the agency — called Western Europe and Strategic Partnerships — was spying on Japan in an effort to gather intelligence about its foreign policy and trade activities. Moreover, as of July 2010, the NSA had obtained domestic court orders enabling it to conduct surveillance on U.S. territory of Japanese officials and the Bank of Japan, which has offices in Washington, D.C., and New York City. The NSA’s covert eavesdropping operations give it an insight into the Japanese government’s private negotiations and dealmaking. As was the case in late May 2007, during a secret meeting at the luxury Hotel Captain Cook in downtown Anchorage, Alaska.