Hackers breached a server used by popular virtual network provider NordVPN and stole encryption keys that could be used to mount decryption attacks on segments of its customer base.

A log of the commands used in the attack suggests that the hackers had root access, meaning they had almost unfettered control over the server and could read or modify just about any data stored on it. One of three private keys leaked was used to secure a digital certificate that provided HTTPS encryption for nordvpn.com. The key wasn't set to expire until October 2018, some seven months after the March 2018 breach. Attackers could have used the compromised certificate to impersonate the nordvpn.com website or mount man-in-the-middle attacks on people visiting the real one. Details of the breach have been circulating online since at least May 2018.

Based on the command log, another of the leaked secret keys appeared to secure a private certificate authority that NordVPN used to issue digital certificates. Those certificates might be issued for other servers in NordVPN's network or for a variety of other sensitive purposes. The name of the third certificate suggested it could also have been used for many different sensitive purposes, including securing the server that was compromised in the breach.

The revelations came as evidence surfaced suggesting that two rival VPN services, TorGuard and VikingVPN, also experienced breaches that leaked encryption keys. In a statement, TorGuard said a secret key for a transport layer security certificate for *.torguardvpnaccess.com was stolen. The theft happened in a 2017 server breach. The stolen data related to a squid proxy certificate.

TorGuard officials said on Twitter that the private key was not on the affected server and that attackers "could do nothing with those keys." Monday's statement went on to say TorGuard didn't remove the compromised server until early 2018. TorGuard also said it learned of VPN breaches last May, "and in a related development we filed a legal complaint against NordVPN."

VikingVPN officials have yet to comment.

Serious concerns

One of those keys expired on December 31, 2018, and the other went to its grave on July 10 of the same year, a company spokeswoman told me. She didn't say what the purpose of those keys were. A cryptography feature known as perfect forward secrecy ensured that attackers couldn't decrypt traffic simply by capturing encrypted packets as they traveled over the Internet. The keys, however, could still have been used in active attacks, in which hackers use leaked keys on their own server to intercept and decrypt data.

It was unclear how long the attackers remained present on the server or if they were able to use their highly privileged access to commit other serious offenses. Security experts said the severity of the server compromise—coupled with the theft of the keys and the lack of details from NordVPN—raised serious concerns.

Here is some of what Dan Guido, who is the CEO of security firm Trail of Bits, told me:

Compromised master secrets, like those stolen from NordVPN, can be used to decrypt the window between key renegotiations and impersonate their service to others... I don't care what was leaked as much as the access that would have been required to reach it. We don't know what happened, what further access was gained, or what abuse may have occurred. There are many possibilities once you have access to these types of master secrets and root server access.

Insecure remote management

In a statement issued to reporters, NordVPN officials characterized the damage that was done in the attack as limited.

Officials wrote:

The server itself did not contain any user activity logs... None of our applications send user-created credentials for authentication, so usernames and passwords couldn't have been intercepted either. The exact configuration file found on the internet by security researchers ceased to exist on March 5, 2018. This was an isolated case, no other datacenter providers we use have been affected.

The breach was the result of hackers exploiting an insecure remote-management system that administrators of a Finland-based data center installed on a server NordVPN leased. The unnamed data center, the statement said, installed the vulnerable management system without ever disclosing it to its NordVPN. NordVPN terminated its contract with the data center after the remote management system came to light a few months later.

NordVPN first disclosed the breach to reporters on Sunday following third-party reports like this one on Twitter. The statement said NordVPN officials didn't disclose the breach to customers while it ensured the rest of its network wasn't vulnerable to similar attacks.

The statement went on to refer to the TLS key as expired, even though it was valid for seven months following the breach. Company officials wrote:

The expired TLS key was taken at the same time the datacenter was exploited. However, the key couldn't possibly have been used to decrypt the VPN traffic of any other server. On the same note, the only possible way to abuse the website traffic was by performing a personalized and complicated MiTM attack to intercept a single connection that tried to access nordvpn.com.

Not as hard as claimed

The suggestion that active man-in-the-middle attacks are complicated or impractical to carry out is problematic. Such attacks can be carried out on public networks or by employees of Internet services. They are precisely the type of attacks that VPNs are supposed to protect against.

"Intercepting TLS traffic isn't as hard as they make it seem," said a security consultant who uses the handle hexdefined and has spent the past 36 hours analyzing the data exposed in the breach. "There are tools to do it, and I was able to set up a Web server using their TLS key with two lines of configuration. The attacker would need to be able to intercept the victim's traffic (e.g. on public Wi-Fi)."

Note also that the statement says only that the expired TLS key couldn't have been used to decrypt VPN traffic of any other server. The statement makes no mention of the other two keys and what type of access they allowed. The compromise of a private certificate authority could be especially severe because it might allow the attackers to compromise multiple keys that are generated by the CA.

Putting all your eggs in one basket

VPNs put all of a computer's Internet traffic into a single encrypted tunnel that's only decrypted and sent to its final destination after it reaches one of the provider's servers. That puts the VPN provider in the position of seeing huge amounts of its customers' online habits and metadata, including server IP addresses, SNI information, and any traffic that isn't encrypted.

The VPN provider has received recommendations and favorable reviews from CNET, TechRadar, and PCMag. But not everyone has been so sanguine. Kenneth White, a senior network engineer specializing in VPNs, has long listed NordVPN and TorGuard as two of the VPNs to reject because, among other things, they post pre-shared keys online.

Until more information is available, it's hard to say precisely how people who use NordVPN should respond. At a minimum, users should press NordVPN to provide many more details about the breach and the keys and any other data that were leaked. Kenneth White, meanwhile, suggested people move off the service altogether.

"I have recommended against most consumer VPN services for years, including NordVPN," he told me. "[The services'] incident response and attempted PR spin here has only enforced that opinion. They have recklessly put activists' lives at risk in the process. They are downplaying the seriousness of an incident they didn't even detect, in which attackers had unfettered admin LXC 'god mode' access. And they only notified customers when reporters reached out to them for comment."