The revelation that a Russian criminal gang successfully stole more than 1.2 billion passwords has left plenty of security professionals scratching their heads over whether the hack was legitimate — or if it even matters.

The New York Times reported on Tuesday that a U.S. security firm revealed that a Russian criminal gang has stolen over 1.2 billion usernames and passwords from websites across the globe.

The report was the result of an 18-month investigation from suburban Milwaukee-based security firm, Hold Security. Hold claims that it has discovered "what could be arguably the largest data breach known to date."

This is usually the point at which security experts weigh in about the vulnerabilities that allowed this kind of breach to happen, and for websites to write articles about what passwords you should change and why using a password manager is more important than ever.

This time, however, security researchers are responding to the revelation with a heavy dose of skepticism — and lots of unanswered questions.

Questions about Hold Security's motives

Why? Well, that's largely because of the way Hold Security disclosed this information thus far.

Almost as soon as the New York Times article went live, journalists at Forbes and The Wall Street Journal, pointed out that Hold Security is will let users (presumably, web services) find out if their data was part of the breach for just $120 a year.

Using a security threat to sell products and services is something that security companies do all the time. It's how the security game works. The problem for some members of the security community, however, is twofold.

Over a Billion Passwords Stolen? http://t.co/us385QpMGe < Bruce Schneier is sceptical of the whole Hold Security brouhaha. And if he is…— Dominic Wellington (@dwellington) August 7, 2014

First, Hold Security isn't actively known in the community (at least, by the name Hold Security). Many researchers, even in the Milwaukee area, have never heard of the company or its founder, Alex Holden.

According to a commenter on famed security guru Bruce Schneier's blog, the company didn't even have an active website until the day the story hit. That doesn't necessarily mean anything, but it is worth noting that the web presence for a security company that managed to uncover theft on this level was virtually invisible until just two days ago.

Not every security researcher is concerned. Respected security journalist and security Brian Krebs, who was the first to break the news about the Target credit card hack, has vouched for Holden and his work. In fact, Krebs says that Holden's research was central to several of his big scoops, including the Adobe password breach.

It is worth noting that Krebs is listed as a trusted advisor to Hold Security.

Questions over data validation

Although the backing from Krebs has significantly mitigated fears that Hold Security could just be a fly-by-night security firm that is actually just trying to cash-in on the next security panic — or worse, that it could be trying to steal user information — it hasn't soothed all experts doubts about the hack itself.

It also doesn't change the fact that Hold Security's "service" to check to see if your data is part of the breach seems poorly thought out.

"We will never ask for your password. Please enter your password" says firm that revealed "billion password" breach http://t.co/gfPJv6heYK— Graham Cluley (@gcluley) August 6, 2014

Independent security analyst Graham Cluley echoed the sentiments of many in the security, writing "something just didn't 'feel right.'"

As Cluley explains, Hold Security wants users to sign-up for what it calls the Consumer Hold Identity Protection Service (CHIPS). This is kind of like a credit monitoring service, except instead of auditing your credit for identity theft, it audits your passwords and email for security theft.

Despite claiming that Hold Security will never ask users for their passwords, the form to submit information to be checked against the database does exactly that.

This is problematic, Cluley writes, for many reasons:

For one thing, what if the computer the user is typing on has keylogging malware in the background –- isn't it going to be trivial for malicious hackers to scoop up the victim's most sensitive passwords as they are entered on this web form? Or what about the possibility of bad guys creating phoney versions of this webpage, specifically with the intention of nabbing users' passwords? But most fundamentally, you should never encourage users to enter passwords for website X into an entirely different website, even if the intention is not to transmit them unencrypted to a third-party site. Isn't this the firm that just warned the world about a huge number of stolen credentials? And here it is coaxing users to behave in a way which is clearly unsafe.

Stating that a service will never ask for a user's password, only to immediately ask for a list of passwords (with the promise that the passwords will be hashed and encrypted upon submission), makes little sense. Even if nothing suspect is going on, the phrasing is completely tone deaf.

Tomorrow in security news: HoldSecurity gang building database of SHA256-hashed, unsalted passwords.— Martijn Grooten (@martijn_grooten) August 6, 2014

Even though the code Hold Security is using looks solid, that doesn't mean submitting passwords is ever a good idea.

I had a quick glance at the site's source code and couldn't spot any easy bugs https://t.co/NT0bXKJYf5 And I know, in theory it's kind of OK— Martijn Grooten (@martijn_grooten) August 6, 2014

Don't think submitting your passwords to Hold Security is adventurous enough? There is a plain HTTP version of the Submit page too.— Martijn Grooten (@martijn_grooten) August 6, 2014

You can't fault members of the InfoSec community for laughing at the idea that the best way to test data against the stolen cache is to collect encrypted passwords through a web form.

if you send me 1.2 billion mars bars i will match them against the mars bars i have here with me— Chris Boyd (@paperghost) August 7, 2014

Even if the data is real, it might not matter

The real issue with this security story, however, is that there just isn't enough information about the importance of the data allegedly captured.

On the surface, 1.2 billion usernames and passwords seems mammoth, but is it?

Some good points on the 1.2bn stolen records http://t.co/JjDU3AJGZd (HT @cryptoron) FTR: 1.2bn email addresses isn't that much.— Martijn Grooten (@martijn_grooten) August 7, 2014

As The Verge notes, there is a very good possibility that even if 1.2 billion usernames and passwords were stolen, that doesn't necessarily mean the data is that valuable.

The data wasn't allegedly amassed from one service, but from hundreds (or thousands). Moreover, Hold Security says that the cyber gang previously bought data from other other hacks. It never explicitly details if the 1.2 billion figure is from brand new attacks or if it includes previous breaches too.

If it is the latter, the data is instantly less interesting, because it has already been out in the wild. If data from previous attacks is part of the password cache, it also suggests that at least a portion of the passwords are stale and of no use.

The Verge points out that the focus on 1.2 billion password attempts to conflate quantity with quality. But that isn't a fair assessment.

The value of a bunch of passwords for defunct web forums or MySpace circa 2006 isn't necessarily the same as getting a few hundred million Facebook logins.

Ultimately, it may not matter

Even if we assume that all the data from this attack is new — and that some of it is valuable — it still might not really matter.

Bruce Schneier sums it up best writing that this entire incident is "evidence of how secure the Internet actually is":

We're not seeing massive fraud or theft. We're not seeing massive account hijacking. A gang of Russian hackers has 1.2 billion passwords — they've probably had most of them for a year or more — and everything is still working normally. This sort of thing is pretty much universally true. You probably have a credit card in your wallet right now whose number has been stolen. There are zero-day vulnerabilities being discovered right now that can be used to hack your computer. Security is terrible everywhere, and it it's all okay. This is a weird paradox that we're used to by now.

My credit card information has been stolen four times in the last three years because of poor security practices on the part of Sony, Target, a web hosting company and my neighborhood ATM machine. I fully expect to have to get a new card at least four more times in the next three years.

Passwords for various forums and defunct Web 2.0-era startups attached to my email address proliferate the Internet. I get at least 5 attempts to break into my Dropbox account per week.

Still, I feel comfortable buying things online, using cloud services and shopping at a multi-national discount chain. Why? Because I have come to expect security incompetence from everyone and everything.

Rather than hoping my information won't be hacked, I go about my business with the expectation that it will. That's not to say that someone breaking into one of my main email accounts or my bank wouldn't still be devastating. But I do what I can to mitigate that possibility by using secure, unique passwords on important sites, two-factor authentication and absolutely no duplicate passwords on logins that can be tied to another service such as email, Facebook or Twitter.

There are still lots of unanswered questions about this Russian cyber gang, and I'm inclined to believe that the data is real but the threat is overstated. But the bigger question — is our information safe? — was answered a long time ago.

No, it's not safe. But as Bruce Schneier says, that's OK.