Black Hat In-Depth A design flaw in Intel's processors can be exploited to install malware beneath operating systems and antivirus – making it tough to detect and remove.

"It's a forgotten patch to a forgotten problem, but opens up an incredible vulnerability," said Christopher Domas, a security researcher with the Battelle Memorial Institute, who revealed the hardware bug at the Black Hat conference in Vegas last week.

The blunder was introduced in 1995, in the Pentium Pro. It is hardwired into the silicon, and has been staring kernel-level programmers in the face for years.

It allows smart hackers to run rootkit code at the very lowest level on the computer, out of reach of the operating system, its applications, and even the hypervisor. This means the rootkit can, among other things, silently monitor and record the user's every keypress, mouse click, and download.

Efforts to detect the rootkit and eradicate it from a computer can be blocked, or hampered, by the malware itself. A nightmare, in other words.

The good news is that Intel spotted the howler in its processor blueprints, and corrected the issue: chips built from January 2011 and onwards (Sandy Bridge Core CPUs and later) are not affected. Also, operating systems can mitigate against the security hole at the hypervisor level, thus protecting themselves from miscreants exploiting the design flaw, according to infosec boffin Jacob Torrey.

Domas reckons there are hundreds of millions of Intel processors permanently vulnerable – they cannot be updated to fix the problem.

A trusty Linux box from the previous decade that's still plodding away, an office of old PCs, or a relative's aging laptop, are the sorts of computers at risk of attack via this vulnerability. Your fancy new gaming rig, your virtual machine in the cloud or on your workstation, and that office PC bought two years ago in the last refresh, are immune.

The design flaw is super-interesting. Even though today's chips do not feature the engineering gaffe, it's worth studying so lessons can be learned. Essentially, it is possible to execute arbitrary code in the processor's all-powerful, and normally locked away, System Management Mode by abusing an obscure feature in the CPU, and we'll explain what that means.

Layers! Onions have layers. Computers have layers

Intel-compatible processors separate software into what's called rings of privilege, or rings of protection. Lower rings manipulate and control the upper rings, and the upper rings are forbidden from interfering with the lower rings.

Normal applications run in the uppermost unprivileged ring, aka ring 3, and they can't directly affect the underlying system. Ignore rings 2 and 1; virtually nobody uses them.

Your operating system runs in ring 0. Apps in ring 3 cannot directly meddle with the OS and compromise it because the OS is protected in ring 0.

Then there's ring -1, which is where the hypervisor lives, if you're using one. The hypervisor looks after one or more operating systems that live in ring 0. The hypervisor, being in ring -1, is protected from the operating systems running in ring 0. This stops a guest operating system from taking over the host machine.

Your PC's hidden creepy janitor

Finally, under all that, there's ring -2, which contains Intel's System Management Mode (SMM) software. This is untouchable from all the other rings, and is loaded from the motherboard firmware before any operating system or hypervisor is started. The SMM works invisible to the other rings. It is like a hidden creepy janitor, doing little background jobs and keeping the hardware ticking over without the other rings realizing.

For example, the SMM code controls the power to the motherboard hardware when the user wants to put her computer to sleep or wake it up. It emulates a PS/2 keyboard when only a USB one is plugged in. It quietly deals with chipset errata – in other words, if an operating system or hypervisor attempts to access buggy motherboard hardware, the SMM will intercept the request and make it all work as expected in software.

All this happens transparently to the other rings: code running in SMM is invisible to the rest of the computer, and it has complete control over the hardware and firmware. It is the real sysadmin of your PC. Taking control of this janitor means taking control of everything.

Where the janitor lives and works

The SMM code reserves a small chunk of your PC's RAM to store its variables and other private data, and all other rings are forbidden by the motherboard hardware from accessing it. That chunk starts at 0x1FF80000, or half way between the 511MB and 512MB marks in your physical RAM. Any attempt to access that portion of memory from software running outside the SMM is blocked by the Memory Controller Hub on your motherboard.

And where it all goes wrong

Back in around 1993, Intel introduced its Advanced Programmable Interrupt Controller (APIC), which, as its name suggests, manages interrupts coming into a processor. Interrupts are electrical pulses generated by the hardware telling the CPU to stop what it's doing, and sort out this urgent thing instead. It might be a countdown timer that's hit zero, or a hard drive finishing a data transfer command, etc. Most of the time, a driver is ultimately summoned to deal with the interrupt before the CPU continues with what it was doing.

The APIC is a split design: there's a local APIC for each processor core on the motherboard, and usually one IO APIC. The IO APIC hooks up with the hardware, and routes interrupt signals to the local APICs, which decide whether or not to interrupt their core and also pass messages between cores.

These local APICs were introduced as discrete chips (the 82489DX). Starting with the Pentium P54C in 1994, the local APICs were built into the actual processor. The local APICs' control registers – internal variables that configure how they work – were mapped into the processor's physical memory map starting from 0xFEE00000 and 0xFEE01000. That means operating system software could find and talk to a core's local APIC at those memory addresses.

When the Pentium Pro (a P6 family chip) arrived in 1995, Intel allowed kernel-level developers to reprogram the local APIC so that it would appear elsewhere in physical memory. This was handy for moving the local APIC out of the way of low-level software that expected to use that high 0xFEE00000 address for something else.

By using the wrmsr instruction, the operating system developer can configure a processor core to move its local APIC to anywhere in memory. Just write the new physical memory address to the processor's model specific register 0x001b.

Anywhere in memory, you say?

Yes, anywhere in physical memory. Like say, where the SMM code stores its hidden private data.

By mapping the local APIC to 0x1FF80000, or thereabouts, it will overlap the SMM's private chunk of RAM. Now when the SMM is triggered into action by a special interrupt called an SMI, the CPU will stop what it's doing, switch to the SMM in ring -2, and execute its code. While running, the SMM's interrupt handler will want to use its private data in memory – except, it won't be accessing its hidden chunk of RAM, it'll be reading from the local APIC's internal registers instead. And we can control those registers, and use them to feed specially crafted data into the SMM to hijack it.

The SMM code has no idea this is happening; it is oblivious to the deception taking place. Here's how it'll work, step by step, taken from Domas' slides [PDF]:

1. Here's the processor, its local APIC mapped at the default 0xFEE00000, and the SMM's private data at 0x1FF80000

2. A SMI comes in, the processor switches to the SMM in ring -2, and it reads from the memory address 0x1FF88000, which is inside its private RAM space...

...And the chipset allows the read access, so the SMM code gets the data it wanted. All working as expected

3. Now let's call the magic wrmsr instruction, and move the local APIC to 0x1FF88000, overlapping part of the SMM code's private area

4. Let's trigger an SMI by poking a hardware register – in this case, writing zero to IO port 0x00b2

5. The processor switches to the SMM and runs its code. It tries to read from the memory address 0x1FF88000, which should be inside its private RAM space...

6. But ah-ha! Instead, it reads from a control register inside the local APIC. We've made it read data of our choosing, soon power will be ours

Turning this trick into something that can be exploited

So now we've seen how it works: force the powerful SMM code, the sysadmin of your PC, into reading from and writing to memory we more or less control. The next steps are easy, right? No. The local APIC's configuration registers are all over the place: there are huge gaps in its memory map where we can't control the contents. Reading from it returns a zero more than 99 per cent of the time.

Effectively, all we've got a 4KB page of zeroes that we can slap over the SMM code's private memory. Writing to it is pointless. Domas called it a memory sinkhole. The situation looked hopeless.

So Domas looked through Intel's sample SMM code, which is provided to firmware vendors to bake into motherboards. It turns out that pretty much all vendors use Intel's template SMM code.

This template code relies on a crucial data structure, the size and address of which is stored at 0x1FF8A000 in memory: a structure called the Global Descriptor Table (GDT). This table is a throwback from the 1980s. All operating systems need a GDT to do anything useful, because it tells the processor where data and executable code, among other things, are allowed in memory.

When the SMM code is entered from an SMI interrupt, it loads its GDT into the processor by giving it the size and address at 0x1FF8A000. By positioning the sinkhole over where the SMM code expects to read its GDT pointer, the processor will read zero. Before triggering the interrupt, you just make sure a table of our own devising is placed at address zero, and the CPU will load our GDT in SMM mode.

After that, we can redirect execution to somewhere more comfortable in memory while remaining in ring -2. Now we're running the show and not the hidden janitor. We can install our rootkit permanently in the firmware, so even if the hard drives are wiped and the apps and operating system reinstalled, it can be revived.

Exactly how this last part is done, we'll leave as an exercise for the reader. Domas has provided some sample code showing how to reprogram the APIC's physical address.

Fixes and mitigations

Again, this is fixed in Intel chips made from January 2011; you can't move the local APIC over the SMM's protected area. Virtual machines, whether they are in the cloud or on your desktop, cannot exploit this vulnerability (unless your virtual machine manager is braindead insecure and allows guest operating systems to remap real hardware). Techniques, such as monitoring the processor cycle counter, can be used to sniff out hidden rootkits: cycles stolen by the stealth malware will show up to application software.

You must have root or administrator-level access to a machine to exploit this chip bug; a malicious driver, or a program exploiting a privilege escalation flaw in an operating system, can abuse this Intel hardware vulnerability (assuming it's not running inside a virtual machine.)

So the sky is not falling in, but it is rather irritating, and people should be warned. As said earlier, what's truly at risk is old office PCs kicking around, or that loyal pre-Sandy Bridge Linux server you can't be bothered to decommission and replace.

Domas hasn't tried the attack on AMD processors. El Reg hopes the chip biz also spotted the design flaw. Intel has issued software fixes for its server motherboard families S5500HC and S5500HCT, and workstation board family S5520SC, to mitigate "a method that enables malicious code to gain access to System Management Mode (SMM)."

"There are hundreds of millions of computers out there that can't be fixed," Domas told the Black Hat conference. "Intel has been great about this, and published some firmware updates, but really, for some systems, this is unpatchable." ®

Additional reporting and technical info by Chris Williams. If you like this sort of in-depth coverage, drop us a note with ideas for more.