An online encryption method widely used to protect banking, email, e-commerce and other sensitive Internet transactions is not as secure as assumed, according to a report issued by a team of U.S and European cryptanalysts.

The researchers reviewed millions of public keys used by websites to encrypt online transactions and found a small but significant number to be vulnerable to compromise.

In most cases, the problem had to do with the manner in which the keys were generated, according to the researchers. The numbers associated with the keys were not always as random as needed, the research showed.

Therefore, the team concluded, attackers could use public keys to guess the corresponding private keys that are used to decrypt data -- a scenario that was previously believed to be impossible.

"This is an extremely serious cryptographic vulnerability caused by the use of insufficiently good random numbers when generating private keys" for HTTPS, SSL and TSL servers, said Peter Eckersley, senior technologist at the Electronic Frontier Foundation. The EFF contributed data for the research.

"We are presently working around the clock to inform the parties whose keys are vulnerable and the [Certificate Authorities] that issued certificates for them, so that new keys can be generated and the vulnerable certificates can be revoked," he said.

The research was originally scheduled to be released later this year, but became public knowledge in a New York Times story Tuesday

.

Public key cryptography is the fundamental encryption system used to protect Internet transactions. It involves the use of a public key to encrypt data and an associated private key to decrypt it.

For instance, when a user logs into a banking website or a secure e-commerce site, the transactions are encrypted using the site's public key. The data can only be decrypted by the site owner using the corresponding private key.

The public keys are typically embedded in digital certificate that are issued by so-called Certificate Authorities. In theory, it's impossible to guess the make-up of a private key, and no two public/private key pairs are ever the same.

In reality, though, not all keys are generated securely, according to James Hughes, an independent U.S.-based cryptanalyst, Arjen Lenstra, a professor at the Ecole Polytechnique Federale de Lausanne in Switzerland, Maxime Augier, a doctoral student, and three other researchers.

The researchers studied 6.6 million public keys generated using the RSA algorithm, and found that 12,720 were not secure at all and 27,000 others were vulnerable.

"The secret keys are accessible to anyone who takes the trouble to redo our work. Assuming access to the public key collection, this is straightforward compared to more traditional ways to retrieve RSA secret keys," the researchers wrote.

The keys inspected by the researchers were collected from several public databases, including one maintained by the EFF.

Eckersley said hackers could relatively easily take advantage of the weakness by assembling a similar database of public keys and replicating what the researchers did to identify the vulnerable keys.

Bruce Schneier, a noted cryptographer and the author of the Blowfish encryption algorithm, said the research findings are significant, but more information is needed to understand the full scope of the issue.

"It is a random number generator problem, but [the research] doesn't really talk about where it came from," Schneier said.

It's similar, he said, to saying that there are 10,000 people with bad locks but not providing details on who those locks belong to or where they are located.

The random number problems identified in the research could be accidental or have been deliberately introduced, by someone seeking to snoop in on encrypted communications, he added.

Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan, or subscribe to Jaikumar's RSS feed . His e-mail address is jvijayan@computerworld.com.