This was inspired by a talk ( "Attacking the Vista Heap" from Ben Hawkes ) that I heard at Ruxcon . Some of the attacks that Ben Hawkes described depended on deactivating security features in Microsoft's heap implementation ahead of overwriting heap data structures. Amusingly, most of these attacks work because Microsoft disables aborting on internal consistency check failure by default.

OpenBSD's heap implementation is already the best security-wise. Thanks to prior work by otto@ and tedu@ - it has random "canary" values (similar to the propolice stack cookies) to detect overwrites of critical structures, it deliberately introduces randomness in certain operations to make them non-deterministic and checks its internal consistency in various ways and will (by default) abort if there are any problems.

This work implements two new measures to harden the allocator code:

First, it moves all the runtime options such as guard pages, use-after-free protection, and (especially) abort-on-error into a structure that is made readonly via mprotect(2) after it has been initialised. Under normal operation, the malloc.c code will never modify these values once they have been set so any attempt to write to it is a bug or an attack and the mprotect will ensure that doing so will result in a SEGV.

The remainder of the patch implemets additional protection on the main internal metadata structure of the heap implementation: struct dir_info. This patch allocates this structure using mmap(2), which on OpenBSD returns a randomised address. This alone is a good measure, since the structure will no longer be at a predictable address relative to other objects. It also allocates a "guard page" above and below the structure that are set to PROT_NONE via mprotect(3). Any attempt to read or write to an address inside either guard page will result in a SEGV, so the structure will be quite difficult to overflow - an attacker has to guess the address quite precisely.

The diff as committed implemented a 3rd protection controlled by a new 'L' flag for MALLOC_OPTIONS and /etc/malloc.conf: mprotecting the metadata structure PROT_NONE when we leave the allocator code and remapping it read/write when we enter it. This would render it basically impervious to attack, since an attacker wouldn't be able to clobber no matter how accurately they guessed and wouldn't be able to read off copies of the master canary values that they would need to clobber dependant structures. This part of the patch is likely to be removed - it defended against attacks that are highly unlikely to succeed in real life (go ahead, prove me wrong!), was too slow (~20% worse) to turn on by default and, as the Windows Vista example neatly illustrates, security options that are off by default aren't really much use to our users.