A recent statement from anti-malware and threat-detection software manufacturer PC Tools claims that Windows 2000 is a more secure operating system than Windows Vista. The company's claims, as covered by Infoworld, have attracted a good deal of coverage—no one wants to miss out on a good Microsoft bashing—but an examination of the company's methodology raises serious concerns about the validity of the conclusions.

According to company CEO Simon Clausen, "recent research conducted with statistics from over 1.4 million computers within the ThreatFire community has shown that Windows Vista is more susceptible to malware than the eight year old Windows 2000 operating system, and only 37 percent more secure than Windows XP." This certainly sounds dire, and at first glance, the company's numbers back its statements up. Data reports from the company's ThreatFire security program state that Vista allowed an average of 639 threats per 1,000 computers "through." Through, in this case, presumably means that the malware in question successfully installed itself and became active.

Windows 2000, meanwhile, was successfully breached by 586 threats, Windows Server 2003 by 478 threats, and Windows XP by a massive 1,021 threats per 1,000 computers. The immediate "conclusions" from these results are twofold. First, Windows Vista, despite all of Microsoft's work and claims to the contrary, is supposedly less secure than the positively ancient Windows 2000. Second, all of us running Windows XP are completely screwed; PC Tools data indicates that Windows XP systems are infected, on average, with 1.02 "threats."

What constitutes a threat, however, isn't clearly explained, and the questions only pile up from there. We know nothing about how many threats were tested, how those threats were chosen, or whether or not the same suite of threats were applied against all of the available machines. Clausen's quote indicates that the company's data was drawn from user statistics, rather than based on rigorous testing, which opens the door for a slew of confounding variables.

We're given no information on whether or not these systems were fully patched or running updated antivirus software. Intermittent Internet access for all surveyed systems is a given, but there's no information on which systems were behind a firewall/router and which weren't. The questions continue ad nauseam; were all of the systems running in Administrator mode? Was Vista's UAC enabled? If UAC was enabled, did a prompt from it count as a threat block? Was Vista SP1 or an RC version of XP SP3 installed? How were multiple malware infections on the same system counted? The first action of many trojans is to download other trojans, and a system can easily end up with 4-5 separate infections in a short period of time.

PC Tools did not respond to a request for comment on our questions.

Studies that control for the above variables are known as security studies precisely because they endeavor to accurately model relative OS security. Documents that fail to take such considerations into account are better classified as PR statements. With Vista security numbers like these, you'd be well advised to invest in a good anti-virus solution, and amazingly enough, PC Tools happens to have a unique, better-than-the-rest behavioral analysis AV product known as ThreatFire.