With Wednesday's release of Mac OS X Lion, Apple has definitively leapfrogged its rivals by offering an operating system with state-of-the-art security protections that make it more resistant to malware exploits and other hack attacks, two researchers say.

Unlike the introduction of Snow Leopard in 2009, which offered mostly incremental security enhancements, OS X 10.7 represents a major overhaul, said the researchers, who spent the past few months analyzing the OS.

The most important addition is full ASLR. Short for address space layout randomization, the protection makes it much harder for attackers to exploit bugs by regularly changing the memory location where shell code and other system components are loaded. Other improvements include security sandboxes that tightly restrict the way applications can interact with other parts of the operating system and full disk encryption that doesn't interfere with other OS features.

“It's a significant improvement, and the best way that I've described the level of security in Lion is that it's Windows 7, plus, plus,” said Dino Dai Zovi, principal of security consultancy Trail of Bits and the coauthor of The Mac Hacker's Handbook. “I generally tell Mac users that if they care about security, they should upgrade to Lion sooner rather than later, and the same goes for Windows users, too.”

Although ASLR made its OS X debut in Leopard, the predecessor to Snow Leopard, its implementation was woefully inadequate because it failed to randomize core parts of the OS, including the heap, stack, and dynamic linker. That meant entire classes of exploits were automatically immune to the protection.

It also prompted many to wonder why Apple engineers bothered to put it into the OS in the first place, or didn't properly implement it with the introduction of Snow Leopard. Windows Vista and Ubuntu, by contrast, added much more robust implementations of ASLR years earlier.

“When they went from Leopard to Snow Leopard, as far as I'm concerned, there really wasn't any change,” said Charlie Miller, principal research consultant at security firm Accuvant and the other coauthor of The Mac Hacker's Handbook. “They might have said there was more security and it was better, but at a low functionality level there really wasn't any difference. Now, they've made significant changes and it's going to be harder to exploit.”

What's more, Lion's refurbished ASLR has been augmented, so that even if hackers clear that hurdle, they'll still have to bypass other new protections. Among them is a sandbox design that shields the most vulnerable and vital parts of the computer from attack. Safari, for example, has now been divided into two processes that separate the browser's user interface and other functions from the part that parses JavaScript, images, and other web content.

Free rein no more

With virtually all browser exploits targeting the way the program parses web content, Apple engineers have tightly locked down the new process, called Safari Web Content. The design is intended to limit the damage that can be done in the event an attacker is able to exploit a buffer overflow or other bug in the browser.

“Now, you end up inside this restricted process that only does the web parsing, and you can't do other things you might want to do as an attacker, such as write files or read a person's documents,” Miller explained. “Even when you get code execution, you no longer have free rein to do whatever you want. You can do only what the sandbox allows you to do.”

Apple has also completely rebuilt its FileVault disk encryption scheme to make it operate at the block level, well below the file level, as was the case previously. As a result, users may now encrypt an entire hard drive rather than only a user directory. It also eliminates a major frustration for many users, because for the first time it works seamlessly with the Mac's Time Machine feature for automatically backing up disk contents. (Previously FileVault users who wanted to restore backup files had to boot off a CD.)

FileVault 2 also includes other improvements, such as the ability to encrypt disk contents when a Mac is put to sleep.

Fanbois, beware

No doubt, Apple deserves kudos for setting a new standard in OS security that Microsoft and Linux distributors would do well to emulate. But it would be a mistake for Mac users to think their OS is invulnerable. As the most recent Jailbreakme exploit made clear, real-world hackers can still find ways to root iPhones and iPads when the devices do nothing more than visit a booby-trapped website.

If iDevices, which contain security protections that go well beyond those found in OS X, can succumb to drive-by downloads, there's no reason Macs aren't also vulnerable.

“Those guys are seriously raising the bar, but no matter how high the bar is, somebody is going to get over it,” said Rich Mogull, CEO of Securosis. “As long as we have even moderately open operating systems, there's always going to be somebody smart enough to get through that stuff.” ®