The Repeated Failure Of The US And UK Governments' 'Add More Hay' Approach To Surveillance

from the sipping-from-a-firehose dept

Recently we wrote about a UK Parliamentary report absolving the UK spy agencies of any responsibility for the failure to stop the killing of a British soldier last year. Significantly, one explanation given for the fact that the UK's MI5 undervalued the threat, despite investigating the men responsible several times, was that it has several thousand suspects under surveillance at any one time, and so it was beyond its capabilities to follow all leads thoroughly.

Of course, that is a consequence of the "needle in a haystack" approach that the US and UK agencies have adopted: collect as much information as possible in the hope that somehow it will be possible to sift through all the irrelevant hay to find the needle. But as an important piece by Coleen Rowley in the Guardian points out, this is not the first time that a "failure to connect the dots" from information to hand resulted in missed opportunities to stop attacks: as an FBI whistleblower and witness for several US official inquiries into 9/11 intelligence failures, I fear that terrorists will succeed in carrying out future attacks -- not despite the massive collect-it-all, dragnet approach to intelligence implemented since 9/11, but because of it. This approach has made terrorist activity more difficult to spot and prevent. She reminds us: The common refrain back then was that, pre 9/11, intelligence had been flowing so fast and furiously, it was like a fire hose, "and you can’t get a sip from a fire hose". Intelligence such as the Phoenix memo -- which warned in July 2001 that terrorist suspects had been in flight schools and urgently requested further investigation -- went unread. She details other instances of recent intelligence failures where having too much information meant that key leads were buried, and opportunities to stop attacks lost. In other words, US and UK agencies have over ten years of bad experiences with the "collect it all" and "needle in a haystack" approach to intelligence gathering, and yet it remains their principal response to successive attacks and continuing failures. Rowley concludes: After Edward Snowden described just how massive and irrelevant the US and UK monitoring had become, people started to grasp the significance of the saying: "If you’re looking for a needle in a haystack, how does it help to add hay?" Of course, she's not the first person to make this observation -- for example, Techdirt pointed this out over a year ago. But as someone who has worked for the FBI, and been privy to many secret details of surveillance operations, Rowley speaks with a special authority. Her article in the Guardian is well-worth reading -- especially by all those who continue to advocate the disproportionate and ineffective "add more hay" approach to keeping the public safe.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community. Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis. While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: gchq, haystacks, information overload, needles, nsa, surveillance