Information security boffins have pilloried Verizon's latest data breach report, suggesting its list of top security vulnerabilities do not represent reality.

The 2016 Data Breach Investigations report [PDF] is Verizon's ninth in the series drawing on a wider pool of data including some 100,000 security incidents and 2260 data breaches last year. It includes case load data from Verizon and some 50 other organisations including computer emergency response teams.

Among its findings is that attackers are popping boxes and siphoning off data with days or minutes, using multiple exfiltration paths, and with most exploiting weak or stolen credentials.

Verizon claims vulnerabilities it compiled into a top 10 list are responsible for 85 per cent of attacks, something security types say is disconnected from reality.

The company has been contacted for comment.

Criticism is still pouring out of security blogs and social media accounts, much of it going into great detail about alleged flaws in Verizon's data set.

Much of the criticism centres on noise and false positives generated by vulnerability scans and intrusion detection systems on which the list is in part based.

Michael Roytman senior data scientist with Kenna Security, which contributed to the vulnerability data with Tenable, Rapid 7, and Qualys, produced another top vulnerability list following a discussion with 451 Research senior analyst Adrian Sanabria who offered what he saw as flaws in the Verizon report.

Wait, FREAK (CVE-2015-1637) is one of the top 10 exploited vulnerabilities in 2015? For real? https://t.co/GPLjVmSaUP — Andreas Lindh (@addelindh) April 27, 2016

He says the list was built by pairing CVE vulnerability information with intrusion detection system data, meaning an entry was logged if a system was both exposed to a flaw and contained a logged intrusion.

Detractors say this approach is flawed, however.

Security man Brian Martin of attrition.org wrote that the methodology missed half of disclosed vulnerabilities by relying on CVE and that the detection signatures used would likely produce a strong bias.

His points were taken up by Errata Security chief executive officer Robert Graham who criticised the use of intrusion detection system data claiming Verizon did not sufficiently cleanse the sets of false positives and noise, and joining others in ridiculing the report for including the infrequently-exploited FREAK man-in-the-middle vulnerability in its top 10.

"[FREAK is] a man-in-the-middle attack. In other words, you can't attack a web server remotely by sending bad data at it," Graham says, adding that "... even the NSA does not have sufficient compute power to crack as many keys as the Verizon DBIR claim were cracked."

"Verizon didn't pay attention to the details. They simply dumped the output of an IDS inappropriately into some sort of analysis. Since the input data was garbage, no amount of manipulation and analysis would ever produce a valid result."

Trail of Bits chief executive officer Dan Guido offered similar criticism, dubbing the vulnerability list a "travesty" on what is otherwise a huge and valuable source of data for the security community.

"The report’s most glaring flaw is the assertion that the TLS FREAK vulnerability is among the top 10 most exploited on the internet," Guido says.

"No experienced security practitioner believes that FREAK is widely exploited."

Guido says Verizon should have run their data by experienced offensive security hackers which would have caught the claimed flaws.

FoxGloveSecurity penetration tester Justin Kennedy broadcast shared criticisms pointing out alleged flaws in that denial of service is listed as a major attack vector.

Kennedy offered his own list based on his experience as an offensive security wonk, basing his results on some 200 customer security tests made by himself and his team in the years since 2011.

The vulnerabilities required manual verification, must have "significant impact", and originate from an external penetration test. It included authentication bypasses; internal network access; privilege escalation; remote command execution, and unauthorised access to mission-critical data.

Kennedy's top 10 vulnerabilities SQL Injection; Insecure Authorisation; Insecure Direct Object Reference; Stored Cross-site Scripting; Insecure Authentication; Insecure Password Reset; Guessed Password; Default Credentials; Single Factor Authentication, Insecurely Configured Application Server.

"Some of these categories could be presented differently; however, this is what made the most sense to us," Kennedy says.

He added "... we wanted to look at the impact directly resulting from exploitation of that specific instance of the vulnerability, or the impact of that vulnerability when it was part of a larger exploit chain." ®