The old and insecure MD5 hashing function hasn't been used to sign SSL/TLS server certificates in many years, but continues to be used in other parts of encrypted communications protocols, including TLS, therefore weakening their security.

Researchers from the INRIA institute in France have devised several attacks which prove that the continued support for MD5 in cryptographic protocols is much more dangerous than previously believed.

They showed that man-in-the-middle attackers can impersonate clients to servers that use TLS client authentication and still support MD5 hashing for handshake transcripts. Intercepting and forwarding credentials through protocols that use a TLS channel binding mechanism is also possible.

The same will apply in the future to the SHA-1 hashing function which is currently being phased out from digital certificate signing.

Their attacks are dubbed SLOTH, which stands for Security Losses from Obsolete and Truncated Transcript Hashes, but is also a comment on the overly slow pace of phasing out legacy and insecure algorithms like MD5 from protocols.

MD5 dates back to 1992 and is used for hashing -- the process of taking an input and generating a unique cryptographic representation of it that can serve as an identifying signature. Unlike encryption, which is reversible, hashing is supposed to only work one way. It's built on the premise that no two inputs should result in the same hash, or signature.

If the algorithm allows two inputs to match the same hash, then it is vulnerable to a so-called collision attack. This means that an attacker can take a legitimate file, like a certificate, modify it, and still present it as valid because it has the same hash as the original.

MD5 signatures have been known to be insecure and vulnerable to practical collisions since at least 2005 and their use for signing SSL/TLS server certificates has been phased out. However, support for the algorithm was kept in other parts of the protocol where its use was still considered safe due to other factors.

Most of the encrypted Web is based on server authentication, where the client verifies the server's certificate to make sure that it's talking to the right website and not a rogue one served by an attacker who can intercept and modify network traffic. But there are also implementations of TLS client authentication, where the server verifies the client's certificate, such as with certain banking applications or virtual private networks.

During client authentication, the client signs a hash of the connection handshake transcript with its own certificate. In the case of TLS up to version 1.1 the transcript hash was generated using a combination of MD5 and SHA1, but starting in TLS 1.2 the client and server can negotiate the hashing algorithm based on what they support.

TLS 1.2 allows stronger hash functions like SHA-256 and SHA-512, but also supports MD5. So, if a man-in-the-middle attacker can trick the client to authenticate with a server under his control, he can then impersonate that client to the real server by negotiating MD5 hashing and using what the INRIA researchers call a transcript collision attack.

"We find that the TLS libraries in Java (SunJSSE) and on Akamai Servers support RSA-MD5 signatures for both client and server authentication," the INRIA researchers said in a blog post that explains their findings. "Even implementations that do not advertise support for RSA-MD5, such as NSS (before version 3.21), BouncyCastle (Java before version 1.54, C# before version 1.8,1), PolarSSL/mbedTLS (before 2.2.1), GnuTLS (before version 3.3.15), and OpenSSL (before version 1.0.1f) surprisingly accept RSA-MD5 signatures."

The researchers determined that to find a collision for a client impersonation attack on TLS, the attacker would need to compute 2^39 hashes, which is quite practical and would take several hours on Amazon EC2 instances. In fact, during their proof-of-concept attack, they found the collision is just one hour using a workstation with 48 CPU cores.

Another attack that the researchers demonstrated is credential forwarding. This defeats a mechanism known as tls-unique which binds credentials to the TLS channel used to transmit them. This means that a man-in-the-middle attacker shouldn't be able to capture credentials from a TLS connection and relay them to a legitimate server in order to authenticate, because that would open a different TLS channel.

Channel binding with tls-unique is used in SCRAM, the default authentication protocol for XMPP (Extensible Messaging and Presence Protocol); Token Binding, which is designed to protect HTTP cookies and OAuth tokens; and the FIDO universal authentication framework.

A generic collision that could defeat channel binding would require 2^48 keyed-hash message authentication code (HMAC) computations, the researchers found. In their proof-of-concept attack, they found the collision in 20 days on a workstation with 4 Nvidia Tesla GPUs, but they believe the time can be significantly reduced with parallelization across many more GPUs.

Attacking TLS server authentication, which is what's used in most HTTPS (HTTP Secure) implementations, is also theoretically possible, but fortunately it's much harder than attacking client authentication. That's because the server signature does not cover the whole handshake transcript, at least in TLS 1.2.

That's good news, because according to Internet scans, 30 percent of HTTPS servers are currently willing to send RSA-MD5 server signatures and are theoretically vulnerable. If the attack would have been more practical, it would have been devastating to Web security.

Due to these findings, the editors of TLS version 1.3, which is currently only a draft, have removed all use of MD5 signatures from the protocol. OpenSSL, Mozilla NSS, GnuTLS, BouncyCastle, PolarSSL/mbedTLS already have patched versions available that disable the use of such signatures. Oracle Java clients will be fixed in the next critical patch update.

The researchers have warned that SHA-1, which is also known to be theoretically vulnerable to collisions, could lead to similar problems in the future if it's not removed from TLS 1.2 implementations in a timely manner.

"If practical collision attacks on SHA1 appear, then many constructions in TLS, IKE, and SSH will be breakable," they said. "So, if you can afford to do so, get rid of MD5 and SHA1 in all your protocol configurations."