Will the air bags in your car protect you in a crash? You could drive the car into a bridge abutment to find out, but you probably won't. Instead, you rely on the NHTSA to run crash tests and ensure that car makers are installing the air bags correctly. In the same way, you really don't want to test your antivirus by subjecting it to active malware. Leave that sort of testing to the independent labs around the world; they know what they're doing! Microsoft's free antivirus has been a perpetual sad sack in these lab tests, but recent reports suggest that may be changing.

Located in Magdeburg, Germany, AV-Test Institute is a well-respected lab. The institute takes advantage of its proximity to such scientific organizations as the Fraunhofer Institute for Factory Operation and Automation IFF, the Max Planck Institute for Dynamics of Complex Technical Systems, and the Otto-von-Guericke University of Magdeburg. With this last institution in particular, AV-Test supervises scientific final-year theses and offers course-related internships.

Threefold Evaluation

The researchers at AV-Test realize that there's more to antivirus than detecting and eliminating malware. An effective antivirus also must refrain from erroneously identifying valid programs or websites as malicious. And of course, it can't slow system performance by hogging resources. To address these criteria, they rate each antivirus on protection against malware, low impact on performance, and usability, meaning few or no false positives.

An antivirus can earn six points in each of the three categories, for a maximum total score of 18. In order to receive AV-Test certification, the product needs a total of at least 10 points, and can't have a zero in any of the three criteria.

Sinking Scores

Fully half of the 22 products included in this test scored worse than in the previous round of testing. Losses ranged from 0.5 points to 1.5 points. Even perpetual winner Kaspersky dropped from a perfect 18 to 17.5 points. This time around, only Bitdefender scored 18 points.

I asked AV-Test's CEO, Andreas Marx, if he had any thoughts about why this might be. He pointed out that in general it's not uncommon for protection scores to go up or down a half-point. However, he suggested that the biggest reason for a decline is that this test ran under Windows 7 while the previous test used Windows 8, and "Windows 7 behaves very differently when compared with Windows 8.1 or Windows 10." The next report from this lab will be the first using Windows 10.

Microsoft Makes Good

Other than ESET going from 14 to 14.5 points, Microsoft was the only success story this time around. Its 9.5 point score in the previous test didn't even make the cutoff for certification. This time around, Microsoft brought up its protection score by 2.5 points and its performance score by 2.0 points, for a new total of 14 points overall.

Because Microsoft's antivirus comes free with Windows, AV-Test and other labs treat it as a baseline. A third party product that doesn't do as well as the baseline is in trouble. AhnLab and Comodo barely matched the baseline, while Chinese antivirus Quick Heal dropped below, with 13 points.

Does this mean you can rely on Microsoft's built-in protection and ditch your antivirus? That may be a bit premature. It did raise the protection score by 2.5 points, but even with that its protection score is just three of a possible six points. More than half of the products tested earned the full six points for protection.

On the other hand, Microsoft also did well in the latest report from Dennis Technology Labs. In the past, Microsoft has famously flunked this real-world test, earning below-zero scores. In the latest test, Microsoft managed AA certification, the second-highest rating.

One thing's for sure. The fact that modern Windows versions offer some degree of antivirus protection even for users who are oblivious to the need is a good thing. If that protection starts to approach the effectiveness of third-party products, the end result can only be good for consumers.

Image courtesy of Flickr User ToddABishop.

Further Reading

Security Reviews