Researchers have discovered a potentially catastrophic flaw in one of the Internet's core building blocks that leaves hundreds or thousands of apps and hardware devices vulnerable to attacks that can take complete control over them.

The vulnerability was introduced in 2008 in GNU C Library , a collection of open source code that powers thousands of standalone applications and most distributions of Linux, including those distributed with routers and other types of hardware. A function known as getaddrinfo() that performs domain-name lookups contains a buffer overflow bug that allows attackers to remotely execute malicious code. It can be exploited when vulnerable devices or apps make queries to attacker-controlled domain names or domain name servers or when they're exposed to man-in-the-middle attacks where the adversary has the ability to monitor and manipulate data passing between a vulnerable device and the open Internet. All versions of glibc after 2.9 are vulnerable.

Maintainers of glibc, as the open source library is called, released an update that patches the vulnerability. Anyone responsible for Linux-based software or hardware that performs domain name lookups should install it as soon as possible. For many people running servers, patching will be a simple matter of downloading the update and installing it. But for other types of users, a fix may not be so easy. Some apps that were compiled with a vulnerable version of glibc will have to be recompiled with an updated version of the library, a process that will take time as users wait for fixes to become available from hardware manufacturers and app developers.

“A big deal”

"It's a big deal," Washington, DC-based security researcher Kenn White told Ars, referring to the vulnerability. "This is a core bedrock function across Linux. Things that do domain name lookups have a real vulnerability if the attacker can answer."

The widely used secure shell, sudo, and curl utilities are all known to be vulnerable, and researchers warn that the list of other affected apps or code is almost too diverse and numerous to fully enumerate. Using a proof-of-concept exploit released Tuesday, White was able to determine that the version of the Wget utility he uses to test and query Web servers was vulnerable. He said he suspects that the vulnerability extends to an almost incomprehensibly large body of software, including virtually all distributions of Linux; the Python, PHP, and Ruby on Rails programming languages; and many other things that uses Linux code to look up the numerical IP address of an Internet domain. Most Bitcoin software is reportedly vulnerable, too.

One Linux-based package that's not vulnerable is Google's Android mobile operating system. It uses a glibc substitute known as Bionic and isn't susceptible, a company representative said.

The vulnerability, which is indexed as CVE-2015-7547, was disclosed Tuesday by researchers from Google. In a blog post, the researchers said they stumbled on the vulnerability when one of their SSH applications experienced an extremely serious error known as a segmentation fault each time it tried to contact a specific Internet address. Google engineers eventually figured out that the error was caused by a buffer overflow inside glibc that made malicious code-execution attacks possible and then notified glibc maintainers.

To the surprise of the Google researchers, they soon learned that glibc maintainers had been alerted to the vulnerability last July. They also learned that people who work for the Red Hat Linux distribution had also independently discovered the bug and were working on a fix.

"This was an amazing coincidence, and thanks to their hard work and cooperation, we were able to translate both teams’ knowledge into a comprehensive patch and regression test to protect glibc users," the Google researchers wrote.

They went on to say that weaponized exploits that successfully execute malicious code are "possible, but not straightforward" since they require the bypassing of address space layout randomization and other protections designed to make software more resistant to attacks. To prevent the vulnerability from being exploited maliciously, Google researchers aren't releasing the more advanced exploit they developed. The previously mentioned proof-of-concept attack merely crashes an application so users can figure out if it's vulnerable.

Anyone who is in a position to update should do so as soon as possible. Google's blog post continued:

Google has found some mitigations that may help prevent exploitation if you are not able to immediately patch your instance of glibc. The vulnerability relies on an oversized (2048+ bytes) UDP or TCP response, which is followed by another response that will overwrite the stack. Our suggested mitigation is to limit the response (i.e., via DNSMasq or similar programs) sizes accepted by the DNS resolver locally as well as to ensure that DNS queries are sent only to DNS servers which limit the response size for UDP responses with the truncation bit set.

Meanwhile, Glibc maintainers provided the following additional mitigation details:

Mitigating factors for UDP include:

- A firewall that drops UDP DNS packets > 512 bytes.

- A local resolver (that drops non-compliant responses).

- Avoid dual A and AAAA queries (avoids buffer management error) e.g.

Do not use AF_UNSPEC.

- No use of `options edns0` in /etc/resolv.conf since EDNS0 allows

responses larger than 512 bytes and can lead to valid DNS responses

that overflow.

- No use of `RES_USE_EDNS0` or `RES_USE_DNSSEC` since they can both

lead to valid large EDNS0-based DNS responses that can overflow. Mitigating factors for TCP include:

- Limit all replies to 1024 bytes. Mitigations that don't work:

- Setting `options single-request` does not change buffer management

and does not prevent the exploit.

- Setting `options single-request-reopen` does not change buffer

management and does not prevent the exploit.

- Disabling IPv6 does not disable AAAA queries. The use of AF_UNSPEC

unconditionally enables the dual query.

- The use of `sysctl -w net.ipv6.conf.all.disable_ipv6=1` will not

protect your system from the exploit.

- Blocking IPv6 at a local or intermediate resolver does not work to

prevent the exploit. The exploit payload can be delivered in A or

AAAA results, it is the parallel query that triggers the buffer

management flaw.

Red Hat officials have more information here.

Tuesday's disclosure comes 13 months after researchers reported a separate code-execution bug in libc that took on the name Ghost . The bug responsible for that threat had been patched two years earlier, but most production servers running Linux had never received the update.

It will take time for engineers and researchers to analyze this latest bug, but at first blush, it appears it poses the biggest threat to routers and Internet-of-things devices that use glibc. Those devices rarely contain the types of No-eXecute and ASLR mitigations that come standard on most production servers, making code-execution exploits much easier to develop. What's more, hardware is rarely updated, making it unlikely vulnerable devices will ever be patched. The good news, as people have pointed out on social media and in Ars comments, is that many home routers use glibc alternatives that have yet to be reported as vulnerable. It could take weeks for a complete list of vulnerable devices to become available.

It remains unclear why or how glibc maintainers allowed a bug of this magnitude to be introduced into their code, remain undiscovered for seven years, and then go unfixed for seven months following its report. By Google's account, the bug was independently uncovered by at least two and possibly three separate groups who all worked to have it fixed. It wouldn't be surprising if over the years the vulnerability was uncovered by additional people and possibly exploited against unsuspecting targets.

Post updated throughout to add newly available details.