Google has long struggled with how best to get dozens of Android smartphone manufacturers—and hundreds of carriers—to regularly push out security-focused software updates. But when one German security firm looked under the hood of hundreds of Android phones, it found a troubling new wrinkle: Not only do many Android phone vendors fail to make patches available to their users, or delay their release for months; they sometimes also tell users their phone's firmware is fully up to date, even while they've secretly skipped patches.

On Friday at the Hack in the Box security conference in Amsterdam, researchers Karsten Nohl and Jakob Lell of the firm Security Research Labs plan to present the results of two years of reverse-engineering hundreds of Android phones' operating system code, painstakingly checking if each device actually contained the security patches indicated in its settings. They found what they call a "patch gap": In many cases, certain vendors' phones would tell users that they had all of Android's security patches up to a certain date, while in reality missing as many as a dozen patches from that period—leaving phones vulnerable to a broad collection of known hacking techniques.

"We find that there's a gap between patching claims and the actual patches installed on a device. It’s small for some devices and pretty significant for others," says Nohl, a well-known security researcher and SRL's founder. In the worst cases, Nohl says, Android phone manufacturers intentionally misrepresented when the device had last been patched. "Sometimes these guys just change the date without installing any patches. Probably for marketing reasons, they just set the patch level to almost an arbitrary date, whatever looks best."

The Patch Gap

SRL tested the firmware of 1,200 phones, from more than a dozen phone manufacturers, for every Android patch released in 2017. The devices were made by Google itself as well as major Android phone makers like Samsung, Motorola, and HTC, and lesser-known Chinese-owned companies like ZTE and TCL. Their testing found that other than Google's own flagship phones like the Pixel and Pixel 2, even top-tier phone vendors sometimes claimed to have patches installed that they actually lacked. And the lower-tier collection of manufacturers had a far messier record.

'Sometimes these guys just change the date without installing any patches.' Karsten Nohl, Security Research Labs

The problem, Nohl points out, is worse than vendors merely neglecting to patch older devices, a common phenomenon. Instead, it's that they tell users they install patches that they in fact don't, creating a false sense of security. "We found several vendors that didn’t install a single patch but changed the patch date forward by several months," Nohl says. "That’s deliberate deception, and it's not very common."

More often, Nohl believes, companies like Sony or Samsung would miss a patch or two by accident. But in other cases, the results were harder to explain: SRL found that one Samsung phone, the 2016 J5, was perfectly honest about telling the user which patches it had installed and which it still lacked, while Samsung's 2016 J3 claimed to have every Android patch issued in 2017 but lacked 12 of them—two considered as "critical" for the phone's security.

Given that kind of hidden inconsistency, "it's almost impossible for the user to know which patches are actually installed," Nohl says. In an effort to solve that missing patch transparency problem, SRL Labs is also releasing an update to its Android app SnoopSnitch that will let users check their phone's code for the actual state of its security updates.

A Patchwork of Patching Practices

After averaging out the results of every phone tested for each vendor, SRL labs produced the chart below, which splits vendors into three categories based how faithfully their patching claims matched reality in 2017, focusing only on phones that received at least one patch in October of 2017 or later. Phones from major Android vendors including Xiaomi and Nokia had on average between one and three missing patches, and even major vendors like HTC, Motorola, and LG missed between three and four of the patches they claimed to have installed. But the lowest-performing companies on the list were the Chinese firms TCL and ZTE, all of whose phones had on average more than four patches that they'd claimed to have installed, but hadn't.