America’s National Security Agency (NSA) spends upwards of $25m in a year buying previously undisclosed security vulnerabilities – known as zero days, because that’s the length of time the target has had to fix them – but the large investment may not result in as much of a collection of hacking capabilities as is widely assumed.

Jason Healey, a senior research scholar at Columbia University and director at the Atlantic Council policy thinktank, argues that the true number of zero days stockpiled by the NSA is likely in the “dozens”, and that the agency only adds to that amount by a very small amount each year. “Right now it looks like single digits,” he says, adding that he has “high confidence in this assessment.”

Healey presented the research at the Defcon hacking conference in Las Vegas to a packed crowd on the opening day of the event. “I don’t know if we’ve got the right answer, but we’ve tried to run down every line of evidence that we can.”

The question of quite how many unpatched, undisclosed vulnerabilities the NSA has stockpiled cuts to the heart of a long-running concern the information security community has about the agency’s so-called “dual mandate”: it is in charge of procuring intelligence about the actions of America’s enemies, a goal it often pursues through targeted hacking attacks, which are made easier by having knowledge of useful zero days, but at the same time, it is in charge of protecting the information security of the nation, a role which naturally entails warning vendors about unpatched security vulnerabilities it discovers.

NSA claims its discloses 91% of vulnerabilities to vendors

The same tension exists within the wider American government, Healey says. “You see this tension between these agencies, and the government is certainly not of one mind on this … Until 2010 it doesn’t seem like there was a government-wide policy to handle this.”

Before beginning his talk, Healey asked the audience how many vulnerabilities they thought the NSA had stockpiled: hundreds, thousands, more than thousands or less than hundreds. The straw poll showed roughly even numbers guessing each possibility, something that underscores how little trust there is among hackers at large that the NSA will do the “right thing” when it has knowledge of critical bugs.

While emphasising that the closed nature of the NSA makes it hard to state anything categorically, Healey argues that all the available evidence supports the case that the agency actually has much less than the hundreds or thousands or vulnerabilities some in the audience thought it might.

One key piece of evidence comes from the NSA itself, which in 2015 claimed that 91% of vulnerabilities it procured were eventually disclosed to the vendors whose products were at risk. Of the other 9%, at least some of those weren’t disclosed because they were fixed before they could be, the agency adds.

Similarly, the White House has revealed that in one year since the current disclosure policy was implemented, it reviewed about 100 software vulnerabilities discovered by the NSA to determine if they should be disclose, and “kept only about two”. Healey adds that in the autumn of 2014, he was personally told that every single vulnerability which had come up for review had been disclosed.

‘We don’t have a stockpile of zero days’

Aside from anything else, the figures fit with the comparatively low number of zero days found used in the wild in general. According to security researchers Symantec, just 54 were found through the whole of 2015, “so single digits sounds reasonable”.

Healey also cites Michael Daniel, a special assistant to the president and the US’s cybersecurity coordinator, to support the claim: “The idea that we have these vast stockpiles of vulnerabilities stored up – you know, Raiders of the Lost Ark-style – is just not accurate,” Daniel has said.

The figures don’t include the actions of other agencies, though. As the war between Apple and the FBI revealed, conventional law enforcement bodies also have an interest in securing unpatched vulnerabilities. When the FBI eventually bought one such zero day to break into the iPhone 5 at the heart of its fight with Apple – for a reported $1m – it managed to avoid government regulations about zero day disclosure by arguing that it only bought the use of a tool, not the zero day itself. “To me,” Healey said, “it seems to contravene pretty direct presidential guidance.”

Similarly, they don’t include the actions of other governments. Around 30 are known to stockpile their own vulnerabilities, but only one – Britain’s GCHQ – is anywhere approaching public about their activities. GCHQ announced disclosure of 20 zero days last year.

Healey closed with a plea to governments and to the hacker attendees of the conference: “Normally in warfare if one side disarms themselves all they’ve done is disarm themselves. This is the one area where you can disarm governments, because once that information goes to a vendor, everyone is disarmed.”