Jeff Schechtman: Welcome to Radio WhoWhatWhy. I’m Jeff Schechtman. Some men spend their whole lives seeking out fame and fortune. Others put their nose to the proverbial grindstone each day, do their job, and try and do it well, fairly and with integrity and with passion. They don’t seek fame or fortune, merely the satisfaction of a job well done. This is true on the assembly line or in the highest reaches of corporate America or government or even espionage.

Often when these two kinds of men clash the collateral damage can be substantial. My guest, Bill Binney, was as far back as the 1960s one of the NSA’s most distinguished analysts. He had an almost sixth sense for understanding the mathematics behind patterns of contacts and webs of relationships that would prove to be even more valuable than the content itself.

As his distinguished career with the NSA progressed, he would begin to combine these skills with the evolution of the digital age. It was, and he might disagree with this, the perfect coming together of a man, his talents, and the technology of the time. The problem is his superiors had other ideas, ideas about seeking fame and, more importantly, fortune. The clash would fire up his courage as a whistleblower, but it may also have caused the nation thousands of lives on 9/11 and beyond.

This story, Bill Binney’s story, has recently been told in a new documentary out on Netflix entitled A Good American, and Bill Binney is here today to talk to us about this powerful slice of American history that is still very much a part of our search for safety and for privacy.

Bill Binney, thanks so much for joining us here on Radio WhoWhatWhy.

Bill Binney: Thanks, Jeff, for having me.

Jeff Schechtman: I want to go back to the 1960s, to your early days with Army Intelligence, your early work on cryptography and trying to discern patterns to communications and contacts and networks. It was a time when you really began to evolve the idea that such networks and patterns were almost more important than the content itself.

Bill Binney: A lot of it … When I first got into the business it was in the military, so our basic threat then was the Soviet Union. That was the big threat in the world to us, so that was where I was focused. Most of everything they did was encrypted, so you’re basically looking at relationships in the ether in terms of communications, of just basically contacts.

We were working basically with what we now call metadata. Back then we used to call them just network analysis or net analysis or just analysis of military context basically. That meant you had to look at the relationships and patterns of relationships to try to interpret what it meant. That’s basically how it started. It meant that you could look at a lot of data just on a superficial basis, looking at just the metadata and the relationships, and that’s the kind of approach that transitions right into the digital age too.

Because even if you couldn’t read the encryption, you could still get massive amounts of intelligence out of things, because if you had something that was in clear text you’d have that one item, and you look at the content as you’d be reading it, but it still didn’t give you the perspective of your entire range of activity, whereas if you looked at the metadata you could see your whole community of who you’re interacting with and how often.

You could see things like if you had medical problems what doctors you were seeing. You could basically assume certain types of medical problems, depending on their specialties, things like that. You could see what things you buy and just … It’s similar to what companies are doing today except the companies are looking at the individuals only, and that’s primarily to sell you something. Whereas if you’re looking at intelligence you’re looking at groups of individuals or groups of people who are actively pursuing dope smuggling or money laundering or weapons smuggling or any kind of terrorist type activity or pedophilia or any kind of criminal activity of that nature, but it’s groups of people who are involved at that point.

Jeff Schechtman: Using those methods, you were able early on to ascertain things that were beginning to happen, the invasion of Czechoslovakia, the Yom Kippur War, and even the Soviet invasion of Afghanistan in ’79.

Bill Binney: Once you have the understanding of the patterns that you’re looking at, then it becomes pretty clear what is a real threat as opposed to what is not. That’s basically what intelligence is supposed to do. It’s supposed to give you advanced notice of intentions and capabilities. Unfortunately, today they’ve seemed to have lost that perspective. Now they’re pretty much doing a forensics job, which is a police job, after the attacks.

For example, in terrorist attacks here in Europe or back in the United States, they basically come in after the fact and say, “Yeah, we knew this guy was a bad guy,” or “he had all these connections and we knew he was on our watch list or something and we were concerned about him,” but they weren’t following them close enough to be able to stop the attacks.

The reasons they weren’t was of course because their policy of taking bulk acquisition of data on everybody on the planet, which meant you had to dive into this ocean to try to find the fish, you know? That’s the problem. That’s basically what they’re still doing today and that’s why they’re still having trouble stopping anything.

Jeff Schechtman: Even then when these things became clear to you and some of your colleagues at the time, there was reluctance on the part of higher ups to believe it or to act on it.

Bill Binney: Well, yeah, and part of the problem was we … The way we developed things, we did them very efficiently. That was one of our big mistakes. So It didn’t cost a lot of money. The major managers there at NSA didn’t really like that. It didn’t support a large organization. It didn’t support a big budget. Solving the problem was not their main issue. Also their concern was making sure that the agencies and all their contracts increased year-after-year so they had a bigger budget to manage. That was the big thing for them I think.

Jeff Schechtman: One of the ironies in all of that is that even with the desire to find reasons to spend money and to look for contractors to work with, that as you talk about, the NSA was really ill-prepared for the digital age.

Bill Binney: They were basically fat, dumb and happy thinking the Soviet Union would continue and that would be their major threat all along so they could justify the existence of a large organization like NSA, but when they fell apart there in 1990 they got caught. At the same time, just before that, before the Soviet Union fell apart, the digital age was starting to explode, so in the late ’80s, early ’90s the explosion had already started in terms of cell phones and computers and emails and things like that and the managers at NSA were still thinking in the old mentality, we need to have the Soviet Union around so we have an opposition out there that we can look at as the threat.

They’re doing similar things today, trying to make them look like a threat. It’s an external threat. They’re trying to do the same thing again, but it was the whole concept of we have a threat that will justify our existence so we don’t need to worry about anything else. But when it fell apart they had to look around to find out where can we find another justification for our existence, so that became the international crime, terrorism, and the internet in the digital age.

Jeff Schechtman: You were developing the extension, the digital extension of understanding these patterns that would have come, this program ThinThread, and yet because it wasn’t expensive enough, because it wasn’t really involved with spending huge sums of money, there was a pushback to it early on?

Bill Binney: Yes. Yeah. And it … They were asking for something like $3.8 billion to start a separate program to try to deal with the digital age, so when we came along with a solution like ThinThread that solved that $3,200,000 … That’s all we spent on it, okay? And that was like from the beginning to the point where it was operational at three separate sites 24 hours a day for about a year, so we were clearly demonstrating the capacity and capability to handle all this information.

But that became a threat to the budget request for $3.8 billion, because you go to Congress and ask for $3.8 billion, you have to have a problem. The problem they were alleging was volume, velocity and variety in the internet in the digital age. That was pretty much the problem we had already solved, so they had to get rid of us.

We even knew at the time from staffers in Congress on the intelligence committees that all the companies that wanted the feed on that $3.8 billion were in fact down in the Congress in those committees lobbying to get our program canceled. Without any review by anybody except the conversations behind closed doors in Congress with the contractors they got our program canceled, with the cooperation of the NSA management of course.

Jeff Schechtman: Talk a little bit about how ThinThread worked, how the program was able to do what it did essentially for as little money as it did.

Bill Binney: Yeah. It boiled down to this. The first problem we had was to be able to sessionize data, that is take the internet, which passes data around in packets, and pull all those packets back together to form the original input, like an email or a phone call or chat or something. That would come in and we’d have to reassemble that much like your service provider does now, but we’d have to do that online and then they would look at it.

We succeeded at doing that at fiber optic rates in 1998, and by doing it at that point, now we could look at the massive amount of data. But you can’t take it all in, because if you did you’d be doing like you’re doing today. You would just simply bury everybody with information. They couldn’t see the threats coming that way.

So what we did, we used metadata in the relationships of social networks of people in the world to be able to see into that massive flow of data without looking at the content of the data, but just looking at the metadata. Then we could pull out all the data that was relevant for us to analyze and look at. By knowing the communities or individuals who were involved in certain activities, you could do that.

Also you had developmental rules. You could figure out who else was involved with them at the same time, and could do it all online, so that meant you could filter everything right up front and you never take in data that wasn’t relevant. You’d only pull in the data relevant at the targets you want to analyze and everything else you just let go right by.

That fundamentally gave everybody in the world privacy, not just US citizens, but everybody. Then once you pull that data in, if there are people involved in that that you don’t yet know who are participants in activity, what we would do is encrypt their attributes. Once you encrypted their attributes, nobody even inside NSA could tell who they were. That protected the identities of people.

The third thing we had was an automated process to monitor who came into our network, who looked at the data, where they went, how long they stayed, what they did while they were there. It basically gave us a monitor to be able to see who was doing the right thing, who was doing the wrong thing, and the ability to stop them at any point in time.

Those are the three things they removed from the ThinThread program and then translated that software forward to do the Stellar Wind program and eventually how they spied on the entire world, because there was no … We saw no limit to how much data we could take in and index and manage.

Jeff Schechtman: There was a turning point that happened around 1999 when General Michael Hayden became head of the NSA. Talk a little about that.

Bill Binney: He came in March of ’99 I believe it was, or maybe a little later, but he came in with a concept of outsourcing jobs in NSA, like for example the infrastructure. They had a separate program to do outsourcing of infrastructure, computer management, data management and all the communications lines of NSA around the world.

When they did that, that was like the fundamental foundation for all the activities that we wanted to do, and then they just kept that outsourcing concept going. They even outsourced some language translations and things like that so that … Or functions that they felt could be outsourced.

That fundamentally was moving into an area where contractors were taking over jobs that required some degree at least of experience in terms of intelligence direction and realized what you really needed to do there. When they did that, they were giving it basically to people who had no idea what the concepts of intelligence production are all about, so they lost some capability by doing that.

Then fundamentally they also lost the management capacity of all the data they had, because if you looked at it, Edward Snowden was a contractor, he’s one of many. Martin was another one. Contractors who were really involved in maintaining the knowledge and understanding that’s accumulated inside NSA, that’s a kind of dangerous thing to do, because they work for companies that have many facets and industrial espionage can be easily done then, especially if you’re collecting all the data on everybody in the world, including all the companies in the world.

Then that’s certainly a hazard. It also is a hazard in terms of the compromises that could occur, which we witnessed and are continuing to witness. That was one of the problems of outsourcing.

Jeff Schechtman: One of the things that also happened is that some of these contractors, some of these executives from these outside contractors were brought in at the highest levels of the NSA.

Bill Binney: Yes. Yeah. That’s right. Well, for example in the Trailblazer case initially Bill Black came in as the … He was hired directly in from SAIC to be the … He was a vice-president—

Jeff Schechtman: SAIC was one of the outside contractors?

Bill Binney: Right, and they hired him in to be deputy director of NSA. Then they hired Sam Visner in to be the head of the Transformation Office inside NSA. They managed all the large contracts that NSA was going to let them to outsource and modernize. So when it came time for the Trailblazer program, I mean the first contracts were let to SAIC for about $280 million or something like that in the first year, and then it simply went up from there.

Jeff Schechtman: At one point Hayden comes to you and essentially asks you to spend more money. How can you use more money?

Bill Binney: When they first formed the Transformation Office, a friend of mine was the chief of that office at the time and he knew what we were doing down there, so he sent his deputy down to talk to us about … He actually came down and said, “What could you do if we give you $1.2 billion?” After spending $3 million to do a program, what are you going to do with $1.2 billion? That’s kind of a jump in budget.

We said it’ll take a little time to figure out what to do, and we took a few days and came back and said we could upgrade and modernize the entire world and everything back here at NSA and all that around the world, but we could only spend maybe $300 million of that, so they were a little over that.

He went back and said that’s great, and then he went away, and then he came back down the next week and he said, “Well, you guys did such a good job there with that thing, how about $1.4 billion? What could you do with that?” So he was trying to push money at us to try to spend it, but that was the Transformation Office. Once Hayden found out about that, he basically removed him, so he put a new person … That’s when Sam Visner came in after that to take over. They needed another contractor to take over and do what the contractors needed to do.

Jeff Schechtman: Talk a little bit about how what you were doing paralleled the evolution of digital technology itself and how you were managing that within the context of ThinThread in your program.

Bill Binney: A lot of it, if we could what we would do is look around in the commercial environment and see if there’s any software that would do jobs that we saw we needed to do with our data and so on. A lot of things like using different tools to open up attachments and look at attachments and see what’s in them, those kinds of things, we would take and leverage.

We were pretty cheap about it, so some of these products cost only 37 bucks a copy and we wanted to do a few hundred things all simultaneously, we’d go buy a few hundred of these products and just put them online and substitute that instead of building software from the beginning.

Generally the people in the technology area preferred to develop the technology from the very beginning by themselves, not leveraging what’s commercially available, so that meant their budgets had to be so much bigger, because in order to do that they would spend something on the order of $25 a line of code. That was kind of the average of the cost for a contractor to do a line of code. If you multiplied that, if you had 100,000 lines of code, $25 a line just to do something that you could substitute for it by buying a $37 apiece product, that’s the way we took the … That’s the route we took as opposed to developing the software.

Then we’d just … If we needed to, we had certain software we’d have to build ourselves, which we did, but it was limiting it to that which … Only that which we had to do, not … And we did that simply by looking around at the commercial environment to see what’s there and take advantage of it, leverage it. That’s how we could keep it so cheaply.