Juniper's security nightmare gets worse and worse as experts comb the ScreenOS firmware in its old NetScreen firewalls.

Just before the weekend, the networking biz admitted there had been "unauthorized" changes to its software, allowing hackers to commandeer equipment and decrypt VPN traffic.

In response, Rapid7 reverse engineered the code, and found a hardwired password that allows anyone to log into the boxes as an administrator via SSH or Telnet.

Now an analysis of NetScreen's encryption algorithms by Matthew Green, Ralf-Philipp Weinmann, and others, has found another major problem.

"For the past several years, it appears that Juniper NetScreen devices have incorporated a potentially backdoored random number generator, based on the NSA's Dual EC DRBG algorithm," wrote Green, a cryptographer at Johns Hopkins University in Maryland, US.

"At some point in 2012, the NetScreen code was further subverted by some unknown party, so that the very same backdoor could be used to eavesdrop on NetScreen connections. While this alteration was not authorized by Juniper, it's important to note that the attacker made no major code changes to the encryption mechanism – they only changed parameters."

The Dual EC DRBG random number generator was championed by the NSA, although researchers who studied the spec found that data encrypted using the generator could be decoded by clever eavesdroppers.

ScreenOS uses the Dual EC DRBG in its VPN technology, but as a secondary mechanism: it's used to prime a fast 3DES-based number generator called ANSI X9.17, which is secure enough to kill off any cryptographic weaknesses introduced by Dual EC. Phew, right? Bullet dodged, huh?

No. In Juniper's case there's a problem. The encrypted communications can still be decoded using just 30 or so bytes of raw Dual EC output. And, lo, conveniently, there's a bug in ScreenOS that will cause the firmware to leak that very sequence of numbers, undermining the security of the system.

Also, worryingly, ScreenOS does not use Dual EC with the special constant Q defined by the US government – it uses its own value.

Armed with those 30 bytes of seed data, and knowledge of Juniper's weird Dual EC parameters, eavesdroppers can decrypt intercepted VPN traffic.

Now it gets really spy-tastic

Said eavesdroppers were probably involved in introducing one of the vulnerabilities in the first place. Whoever tampered with some builds of ScreenOS changed just the value of Q. No other code was slipped in; just a new constant. Knowing that value, and how to exploit it with the data leak bug, was all a snoop needed.

In other words, someone saw the data leak bug, and knew that if they controlled Q, they could crack encrypted VPN channels.

"To sum up, some hacker or group of hackers noticed an existing backdoor in the Juniper software, which may have been intentional or unintentional – you be the judge," wrote Green.

"They then piggybacked on top of it to build a backdoor of their own, something they were able to do because all of the hard work had already been done for them," the assistant professor added.

"The end result was a period in which someone – maybe a foreign government – was able to decrypt Juniper traffic in the US and around the world."

Green points out that this is a classic example of why backdoors are a bad idea all round. It's something politicians and law enforcement officials may want to ponder the next time they call for mandatory government access to encrypted communications.

If they are going to build backdoors into encryption, such as by fiddling with the mathematics or sliding in convenient bugs, someone else is going to find the way in. ®