In a revelation that shows how the National Security Agency was able to systematically spy on many Cisco Systems customers for the better part of a decade, researchers have uncovered an attack that remotely extracts decryption keys from the company's now-decommissioned line of PIX firewalls.

The discovery is significant because the attack code, dubbed BenignCertain, worked on PIX versions Cisco released in 2002 and supported through 2009. Even after Cisco stopped providing PIX bug fixes in July 2009, the company continued offering limited service and support for the product for an additional four years. Unless PIX customers took special precautions, virtually all of them were vulnerable to attacks that surreptitiously eavesdropped on their VPN traffic. Beyond allowing attackers to snoop on encrypted VPN traffic, the key extraction also makes it possible to gain full access to a vulnerable network by posing as a remote user.

BenignCertain's capabilities were tentatively revealed in this blog post from Thursday, and they were later confirmed to work on real-world PIX installations by three separate researchers. Before the confirmation came, Ars asked Cisco to investigate the exploit. The company declined, citing this policy for so-called end-of-life products. The exploit helps explain documents leaked by NSA contractor Edward Snowden and cited in a 2014 article that appeared in Der Spiegel. The article reported that the NSA had the ability to decrypt more than 1,000 VPN connections per hour.

"It shows that the NSA had the ability to remotely extract confidential keys from Cisco VPNs for over a decade," Mustafa Al-Bassam, a security researcher at payments processing firm Secure Trading, told Ars. "This explains how they were able to decrypt thousands of VPN connections per minute as shown in documents previously published by Der Spiegel."

The revelation is also concerning because data returned by the Shodan search engine indicate more than 15,000 networks around the world still use PIX, with the Russian Federation, the US, and Australia being the top three countries affected. Last weekend's release of BenignCertain and dozens of other NSA-connected attack tools means even relatively low-skilled hackers can now carry out the same advanced attack. Analysis of the exploit binary shows BenignCertain targeted PIX versions 5.3(9) through 6.3(4). The researchers, however, were able to make the key-extraction technique work against version 6.3(5) as well.

Cisco representatives on Friday declined to comment on the revelation, citing the previously mentioned end-of-life policy. Update: After this article went live, Cisco updated a previously written blog post to report that product security incident response team decided to investigate BenignCertain after all. The team found that Adaptive Security Appliance, its currently supported firewall is not vulnerable; PIX versions 6.x and earlier are affected; and PIX versions 7.0 and later are confirmed to be unaffected.

BenignCertain exploits a vulnerability in Cisco's implementation of the Internet Key Exchange, a protocol that uses digital certificates to establish a secure connection between two parties. The attack sends maliciously manipulated packets to a vulnerable PIX device. The packets cause the vulnerable device to return a chunk of memory. A parser tool included in the exploit is then able to extract the VPN's pre-shared key and other configuration data out of the response. According to one of the researchers who helped confirm the exploit, it works remotely on the outside PIX interface. This means that anyone on the Internet can use it. No pre-requirements are necessary to make the exploit work. The researcher provided this packet capture to show the end result of the attack.

Interestingly, Cisco's Adaptive Security Appliance, the firewall that replaced PIX, contained a similarly critical Internet Key Exchange vulnerability that was fixed three months ago. What's more, during the time the PIX vulnerability was active, firewalls from almost a dozen other providers were similarly vulnerable. While BenignCertain worked only against PIX, it's possible that still-undiscovered exploits were developed for other products.

The key-extraction exploit could be even more powerful when combined with other attack tools in the possession of Equation Group, the elite, NSA-connected hacking team tied to it . Another tool called FalseMorel appears to extract the "enable" password that's required to gain administrative control over the PIX firewall itself. The BenignCertain tool lets attackers know if a given firewall is vulnerable to FalseMorel. BenignCertain, FalseMorel, and more than a dozen other tools were mysteriously published last weekend by a previously unknown group calling itself ShadowBrokers.

"Despite the existence of 0days, these tools seem to be overwhelmingly post-exploitation," security expert Rob Graham, CEO of Errata Security, wrote in a blog post published Thursday afternoon. "They aren't the sorts of tools you use to break into a network—but the sorts of tools you use afterwards."

Graham's comments came before the capabilities of BenignCertain were revealed. Now that they have been documented, it's clear at least some of the tools gave, and possibly still give, attackers an initial foothold into targeted networks.