The Meltdown and Spectre vulnerabilities, first revealed at the beginning of the year, affect pretty much anything with a chip in it. That ubiquity has made the process of releasing patches understandably arduous. Every type of impacted hardware and software requires its own specially tailored solution, and even a fix that works as intended may slow down system processes as a side effect. The bigger issue so far, though, is that some patches have done more harm than good, requiring recalls and sowing general confusion.

A lot of the focus has fallen on Intel, because all of the company's modern chips are impacted, and the company's attempts to patch the vulnerabilities have seen mixed results. Intel shares the hot seat, though, with fellow chipmakers ARM and AMD. Operating system developers including Microsoft, Apple, and the Linux Group have also been on the hook for providing patches. These fixes, though, can inadvertently cause serious problems beyond processing slowdowns, including random restarts, and even the blue screen of death. Spectre in particular is also more of a class of vulnerability than one easily resolvable bug, so it's proven especially difficult to create one-size-fits-all patches for the flaw.

"We've never seen such an expansive bug like this that impacts literally every major processor," says David Kennedy, the CEO of TrustedSec, which does penetration testing and security consulting for corporations. "I was on at least 10 calls last week with big companies and two yesterday explaining what's happening. They have no idea what to do when it comes to patching. It's really causing a mess."

Rocky Rollout

It doesn't help that processor companies downplayed the challenges at first.

Intel memorably said in its first statement about Meltdown and Spectre that, "any performance impacts are workload-dependent, and, for the average computer user, should not be significant and will be mitigated over time." Sounds great, right? In practice, Intel has had to repeatedly step on this initial nonchalance, revealing that its newer processors are also susceptible to patch-related slowdowns, and that it pushed out some patches too soon. On Monday, Intel retracted one of its Spectre patches because of random reboot issues, and suggested that system administrators roll it back or skip it if they haven't installed it already. "I apologize for any disruption this change in guidance may cause," Intel executive vice president Neil Shenoy said in a statement.

'All of this is pure garbage.' Linux Creator Linus Torvalds

Intel's problems have trickled down to other manufacturers and developers as well. For example, the cloud infrastructure company VMWare said on Thursday that it would delay microcode—fundamental code that coordinates between hardware and low-level software—updates because of problems with Intel's firmware patches. Similarly, Lenovo announced last week that it had to withdraw some of the firmware patches it had issued because of stability concerns. Dell joined the fray, pulling certain Spectre firmware patches on Monday. "If you have already deployed the BIOS update, in order to avoid unpredictable system behavior, you can revert back to a previous BIOS version," Dell said in an update to customers.

Linux creator Linus Torvalds criticized Intel's patches for the Linux kernel in a public message board on Sunday. "All of this is pure garbage," Torvalds wrote. "The patches are COMPLETE AND UTTER GARBAGE. ... They do things that do not make sense." (Emphasis his.)

Microsoft, too, has gradually admitted to more vulnerability-related Windows slowdowns. The company also had to pause distribution of its Meltdown and Spectre patches for certain AMD processors two weeks ago, because the updates were causing fatal errors in some machines. For its part, Apple recently had to walk back some of its claims about protections for older operating system versions. On Tuesday, the company released various combinations of Meltdown and Spectre patches for High Sierra, Sierra, and El Capitan.