After last week's pwn2own and pwnium contests, browser security is, once more, a hot topic. The existence of flaws in browsers is nowadays taken for granted: what security researchers are most interested in is the mitigation techniques browsers use to try to render those flaws harmless. Microsoft published a recent blog post discussing some of the new mitigation techniques that will be used in Internet Explorer 10.

The post first addresses existing anti-exploitation measures already used by Internet Explorer. These are a mix of compile-time techniques—Microsoft's compiler injects code to detect some buffer overflows, for example—and runtime techniques—such as the "Data Execution Prevention" that makes it harder to exploit buffer overflows.

What's new in Internet Explorer 10? Technically, nothing. New to Windows 8, however, is a much improved version of ASLR, Address Space Layout Randomization, and Internet Explorer 10 takes full advantage of the new capabilities. ASLR is another mitigation technique, designed to make it harder to take advantage of software flaws.

These techniques are all additive. Before these mitigation techniques, attackers could insert the code they wanted to execute directly into overflowed buffers and trick the program into executing it. With DEP, that became impossible: the buffer could be readable, but not executable.

Attackers came up with a couple of clever techniques around this, "return to libc" and "return-oriented programming." With both of these techniques, the attacker took advantage of the fact that the program's executable and its libraries (DLLs) were already executable. With return to libc, the attacker would use the buffer as a kind of input to one of these executable functions. Return-oriented programming generalizes this technique; here the attacker would use the buffer to jump between lots of small fragments of these executable DLLs.

ASLR in turn is created to combat these techniques; both of them depend on the DLLs being in predictable locations in memory; the attacker includes the addresses of the different pieces of executable code it wants as part of their attack. By shuffling DLLs around in memory, this predictability no longer exists.

Windows Vista was the first Microsoft operating system to include ASLR, and it was essentially unchanged in Windows 7. ASLR is useful, but it has limitations. There are three problems in particular.

The first is that the current version just isn't that random: the number of different locations that a DLL can be loaded to is limited, which means that an attacker doesn't have to guess too many times until he gets it right. For 32-bit programs, a large part of this lack of randomness is due to lack of address space: Microsoft still wants to leave a big contiguous chunk of memory free and available for programs to use, because it's much friendlier to those programs. Fulfilling this desire leaves little room to load libraries into, though, and hence relatively little randomness. Compounding this problem is that Windows uses the same amount of randomness even in 64-bit programs. Thanks to their larger address space, 64-bit programs have much more room to breathe, but Windows doesn't really take advantage of it.

The second problem is that the current version is opt-in. DLLs need to contain some extra information before they can be randomized by ASLR, and they need to signal to the system that they contain this information. While software is increasingly built to include this information—especially software like a browser that's regularly exposed to hostile code—it doesn't have to be.

This becomes problematic in the face of plugins and extensions. All the major browsers enable ASLR for their own code and libraries, but there are widely-used plugins, especially older ones, that don't have it enabled. If even one DLL is loaded at a known, predictable memory address, that can be enough for attackers to exploit a system.

Third, not everything is randomized. Some memory allocations are still predictable to a greater or lesser extent.

Windows 8 has solutions for both of these. For the first, 64-bit processes can now opt in to what Microsoft calls "High Entropy ASLR" (HEASLR), which greatly increase the amount of randomness that can used. Metro-style browsers, including Internet Explorer 10, are 64-bit by default, and Internet Explorer 10 opts in to using HEASLR.

To address the second problem, programs can force all DLLs they load to use ASLR even if they don't tell the system that they are safe to use that way, a feature Microsoft calls ForceASLR. With ForceASLR, an old plugin or extension can't undermine ASLR's exploit mitigation capability, because it can't load at a predictable address. This is again an optional feature, and again Internet Explorer 10 opts in.

For the third issue, Windows 8 will randomize more kinds of memory allocation.

Taken together, these features will make Windows 8 programs (or at least, Windows 8 programs that opt in to using ForceASLR and HEASLR) harder to exploit, which means that they'll make Internet Explorer 10 harder to exploit. It's very likely that the Chrome and Firefox Metro browsers will opt in to these systems too, giving them equal access to the new capabilities.

Windows 7 users aren't completely left out in the cold, either. Although HEASLR is a feature unique to Windows 8, Microsoft will be releasing a patch for Windows 7 that retrofits that operating system with ForceASLR. Internet Explorer 10 will install this patch, enabling it to use ForceASLR on Windows 7. This is arguably the most important new protection, so providing it on Windows 7 is welcome.

Will these changes make Internet Explorer 10 bulletproof? Unlikely. Chaouki Bekrar, CEO of VUPEN—the company that exploited Internet Explorer 9 at pwn2own last week—told Ars that although "Internet Explorer 10 will probably be a very secure browser, much harder to exploit than IE9," he thinks that "white hat researchers will probably find a way to achieve full code execution against IE10."

For example, researchers might find a way of "breaking" the randomness that ASLR produces. In the time before ASLR, it was generally considered harmless to use memory addresses for other purposes, such as ID numbers exposed to scripts, because attackers already knew the memory addresses they cared about, and so the extra information wasn't useful to them. That's no longer the case today: every extra piece of information about where things are in memory reduces the strength of ASLR's protection. If attackers can find a vulnerability that exposes information about memory addresses to scripts, they may be able to bypass even the new, stronger ASLR.

VUPEN has a track record of producing successful exploits that undermine mitigation techniques, and Bekrar believes that the company "will be releasing the first full exploit for IE10 as part of our offensive services, the first day Windows 8 will be released."

The purpose of all these techniques is to raise the bar; to make it harder and more expensive to develop reliable attacks on the browsers. Before, attackers just needed a buffer overflow; today, they might typically use a buffer overflow (or certain other software flaws) in conjunction with a non-ASLR plugin. In the future, they might need a buffer overflow plus an information disclosure bug to allow ASLR to be bypassed. The imposition of restrictive sandboxes means they might even need a kernel elevation of privilege bug, too.

An attacker with sufficient motivation will generally get what they want eventually, by chaining together enough different software flaws and a suitable amount of cleverness and trickery. Thanks to the newly strengthened ASLR, Windows 8 and Internet Explorer 10 will be tough nuts to crack. But they're unlikely to be unbreakable.