The security world's paranoiacs have long cautioned that if a computer falls into a stranger's hands, it shouldn't be trusted again. Now one company's researchers have demonstrated how, in some cases, that maxim applies just as strongly to a class of machine that never touches your hands in the first place: cloud servers.

On Tuesday, researchers at the security firm Eclypsium published the results of an experiment in which they showed that they could, for a certain class of cloud computing servers, pull off an insidious trick: They can rent a server from a cloud computing provider—they focused on IBM in their testing—and alter its firmware, hiding changes to its code that live on even after they stop renting it and another customer rents the same machine. And while they made only benign changes to the IBM servers' firmware in their demonstration, they warn that the same technique could be used to plant malware in servers' hidden code that persists undetected even after someone else takes over the machine, allowing the hacker to spy on the server, alter its data, or destroy it at will.

"When organizations use public cloud infrastructure, they’re essentially borrowing equipment, like buying it used off of Ebay, and it can be pre-infected before they start using it," says Yuriy Bulygin, Eclypsium's founder and a former head of Intel's advanced threat research team. "In a similar way, that equipment can be infected if the cloud service provider hasn't sanitized all its equipment at the deepest level, including the firmware."

Cloud Control

That cloud sanitization problem, Eclypsium's researchers were clear to point out, doesn't effect all cloud servers. A typical cloud computing setup generates every customer's computer as a so-called virtual machine, a kind of sealed aquarium within the computer isolated from the server's actual hardware and other customers' virtual machines on the same box. But everyone from Amazon to Oracle to Rackspace also offers so-called bare metal servers, in which a customer rents and fully controls an entire computer in an attempt to improve performance or, ironically, security. IBM has thousands of enterprise customers who use bare metal machines for everything from video conference hosting to mobile payments to neurological stimulation treatments.

By renting a machine in a bare metal setup, an attacker can get far more dangerous levels of access to components that can carry malware over to that server's next renter. "The problem is definitely worse and much easier to exploit on bare metal services," Bulygin says.

Hackers, both in research and real-world intrusions, have for years demonstrated that the firmware in little-considered chips that control everything from USB drives to hard drives can provide a hidden foothold for malicious code. Those infections can evade all antivirus and even survive a complete wipe of a computer's storage.

Eclypsium's researchers homed in on the firmware of a powerful component in the Super Micro servers, which IBM offers customers of its bare metal cloud computing service, known as a baseboard management controller. The BMC is used to remotely monitor and administrate the server, and it's capable of everything from accessing the computer's memory to altering its operating system. In previous research, Eclypsium has even demonstrated that a corrupted BMC can be used to rewrite the firmware of other components, bricking computers or paralyzing them for a potential ransomware attack.

"Once the firmware is infected, there’s really no way to know if it is still infected or to recover from it." Karsten Nohl, Security Research Labs

In their experiments, Eclypsium's researchers would rent an IBM bare metal cloud server, then make a harmless alteration to its BMC's firmware, simply changing one bit in its code. Then they'd stop renting the server, releasing it back into IBM's pool of available machines for other customers. A few hours later, they'd rent enough servers to find the same exact machine again, identifying it by the serial number of its motherboard and other unique identifiers. They found that despite supposedly being handed a "fresh" machine, the BMC firmware alteration remained.