Cryptographers have cracked the encryption schemes used in a variety of satellite phones, a feat that makes it possible for attackers to surreptitiously monitor data received by vulnerable devices.

The research team, from the Ruhr University Bochum in Germany, is among the first to analyze the secret encryption algorithms implemented by the European Telecommunications Standards Institute. After reverse engineering phones that use the GMR-1 and GMR-2 standards, the team discovered serious cryptographic weaknesses that allow attackers using a modest PC running open-source software to recover protected communications in less than an hour.

The findings, laid out in a paper (PDF) to be presented at the IEEE Symposium on Security and Privacy 2012, are the latest to poke holes in proprietary encryption algorithms. Unlike standard algorithms such as AES and Blowfish—which have been subjected to decades of scrutiny from some of the world's foremost cryptographers—these secret encryption schemes often rely more on obscurity than mathematical soundness and peer review to rebuff attacks.

"Contrary to the practice recommended in modern security engineering, both standards rely on proprietary algorithms for (voice) encryption," the researchers wrote in the paper. "Even though it is impossible for outsiders (like us) to decide whether this is due to historic developments or because secret algorithms were believed to provide a higher level of 'security,' the findings of our work are not encouraging from a security point of view."

The GMR-1 standard uses an algorithm that closely resembles the proprietary A5/2 cipher once employed by cellphones based on GSM, or Global System for Mobile Communications. A5/2 was dropped in 2006 after cryptographers exposed weaknesses that made it possible for attackers with modest hardware to crack the cipher in almost real time.

The problem with a5-gmr, as the cipher in GMR-1 is known, is that its output gives adversaries important clues about the secret key used to encrypt communications, Benedikt Driessen, a Ph.D. student who co-authored the paper, told Ars. By making a series of educated guesses based on a small sample of the ciphertext, attackers can quickly deduce the key needed to unscramble the protected data.

"If the guess is correct and given enough equations, the equations can be solved to reveal the encryption key," Driessen said.

He also faulted the algorithm for performing what's known as clocking separately and generating output equations with a low algebraic degree, flaws that also diminish security.

a5-gmr-2, the cipher used in GMR-2 phones, is also vulnerable to cracking when adversaries know a small sample of the data before it was encrypted. Because data sent over phone networks contains headers and other predictable content, it is possible for attackers to exploit the weakness.

Phones under attack

It's tempting for critics of the satphone standards to seize on the security-through-obscurity approach, which relies on the lack of documentation to prevent attacks. But in fairness to the engineers who designed it, the approach hasn't completely failed. The new crack works only on the data sent from a satellite to a phone, making it possible to retrieve data from only one end of a conversation. What's more, researchers have yet to reverse engineer the audio codecs used by the standards, so eavesdropping on voice conversations isn't yet possible.

"Our claim is, (a) we can decrypt and the codec will be revealed shortly which allows full eavesdropping and (b) we can apply the attack to different channels (fax, SMS) for which we don't even need a codec," Driessen said. Satphones "are vulnerable because the protection-layer is worthless."

Over the past couple of years, cryptographers have gradually whittled away at many of the algorithms protecting data sent by phones. Standards including GSM, DECT (Digital Enhanced Cordless Telecommunications), and GPRS (General Packet Radio Service) have all been targeted. Devices that are vulnerable to the latest attacks include the Thuraya So-2510 and the Inmarsat IsatPhone Pro.

The secret algorithms were analyzed by downloading publicly available firmware used by the phones, disassembling the code, and using some clever techniques to isolate the ciphers. The analysis techniques may prove valuable in exposing weaknesses in other encryption schemes as well.

Note: The original opening image for this story has been updated to reflect reader concerns.