A new study conducted by AV-Comparatives, a well-respected independent testing agency, has closely examined whether 250 security products for Android smartphones are capable of protecting users at all.

The test evaluated whether Android anti-virus products available in the official Google Play store can protect against the 2000 most common Android malware threats of 2018.

Compared to the total Android malware in existence, 2000 is a small number – but the fact that these samples were considered the most commonly encountered means that no anti-virus product worth its salt should be failing to detect them all.

For each and every anti-virus product, for each and every malicious sample, the same real-world methodology was used:

The Chrome browser would be opened and the malicious sample would be downloaded.

The downloaded .apk file would be opened with a file explorer app.

The malicious app would be installed.

The installed app would be executed.

That’s plenty of opportunity, I’m sure you would agree, for a decent Android anti-virus product to intercept, prevent, and detect a malicious file.

And yet, out of the 250 security products that AV-Comparatives tested only 80 managed to detect more than 30% of the in-the-wild malware. The testing agency considers any products that block less than 30% of common Android threats to be “ineffective/unsafe”.

Worse still, some apps were determined to not actually be looking for malicious code at all, but instead looking at the names of packages to see if they were matched those in the product’s approved or disallowed database.

This meant that an attacker could potentially simply rename their app to slip it past a user’s defences.

Things get truly ridiculous when you read that some anti-virus apps even managed to detect themselves as malicious – because their creators forgot to add their package names to the whitelist.

“Some of them do not even bother to add their own packages to their whitelists, causing them to report their own app. If using such an AV app, users can never be sure if any of the other apps on their device are actually malicious, because of the AV app’s “block unless whitelisted” policy. Therefore, we do not consider the protection capabilities of these apps to be appropriate.”

Interestingly, AV-Comparatives discovered that many of the anti-virus apps it classified as “unsafe” had strikingly similar user interfaces, and despite their poor ability were frequently awarded four stars or more in Google Play Store reviews.

Clearly, relying upon reviews in the Google Play Store is not enough to guarantee that the Android anti-virus product you are downloading can be trusted.

A mere 23 of the 250 products tested were determined to correctly detect 100% of the malware samples tested against (yes, Bitdefender Mobile Security was one of those products which successfully passed the test with a perfect score, and caused zero false positives).

These best-performing products were mostly from established names in the anti-virus industry – some of which have been working in the field for many years.

And because of this, AV-Comparatives recommends that Android users only run security products built by well-known brands:

As well as participating in tests by independent test institutes, such vendors will have a professional website with contact information and a privacy policy. It should also be possible to try the app – typically a few weeks’ trial use is allowed – before purchasing. Users can then assess the usability and any additional features of the product. A number of vendors make very effective free versions of their apps; generally these are more likely to display advertising than the paid version, though this is not always the case.

My advice, however, would go somewhat further than that:

Remember that just because an app is in the Google Play store it doesn’t mean it’s any good. Google isn’t testing the anti-virus capabilities of a product before listing it.

The full study can be found on the AV-Comparatives website.