michael barbaro

From The New York Times, I’m Michael Barbaro. This is “The Daily.” Today: Across Silicon Valley, tech companies are pursuing contracts with the U.S. military. Kevin Roose on a cautionary tale of where that can lead. It’s Wednesday, March 6.

kevin roose

So I’m a tech columnist. I spend a lot of time talking with and looking at the big tech companies and where they’re going. And one of the most interesting threads that’s come up in the past year or so is this question of how Silicon Valley should engage with the government, and with the military, specifically. And I think we’re at a point right now where the government is saying, we want the things that you’re building. We want you to be part of our modernization efforts.

michael barbaro

Mm-hmm.

kevin roose

And within some of these companies there’s a growing divide between the management of the companies and what they want for their business, and some of the people actually building this technology — the engineers who are working on these advanced systems that are in hot demand by the military and other government agencies. And we just saw this play out in a pretty meaningful way at Microsoft.

michael barbaro

What happened there, exactly? So last year Microsoft signed a contract with the Department of Defense to use a technology called HoloLens in the Army.

archived recording

Good morning, everyone. A few years ago, we started asking ourselves, could we make things so simple that technology would just disappear?

kevin roose

And HoloLens was originally developed — it’s kind of like a headset for augmented reality.

archived recording

Could we place your digital content right into your world, right into your life, with more reality than ever before?

kevin roose

I saw a demo of this a couple years ago, and they were touting it as the next generation of gaming.

michael barbaro

Mm-hmm.

archived recording

Welcome to a new era of Windows. Welcome to Windows Holographic.

kevin roose

People were wearing HoloLens, and you would be swinging your imaginary sword at imaginary monsters. But it all looked like it was happening right there in your living room.

michael barbaro

Mm-hmm.

kevin roose

And last year, Microsoft signed a contract to develop HoloLens technology for use on the battlefield — to give soldiers the ability to wear these headsets and get information about their surroundings, and to increase the lethality of the soldiers on the battlefield.

michael barbaro

So this technology will specifically be deployed within the military so that American soldiers are better at killing the enemy?

kevin roose

That’s the way it’s described in the contract, yes.

michael barbaro

And so, I’m guessing that this was something that Microsoft engineers objected to?

archived recording

And then there’s Microsoft. They’re receiving backlash from their own employees over new groundbreaking technology.

kevin roose

Well, a few of them did. But they did it pretty vocally.

archived recording

Yeah, so more than 150 Microsoft employees signed a letter demanding that the company cancel a $480 million contract to build a HoloLens for the Pentagon saying they, quote —

kevin roose

“Dear Satya Nadella and Brad Smith: We are a global coalition of Microsoft workers, and we refuse to create technology for warfare and oppression.” And it goes on. And basically, they argue that this crossed the line into weapons development — that this was basically taking something that they had built, and turned it into something that was designed to help soldiers on the battlefield kill other soldiers.

michael barbaro

Not what they decided to do.

kevin roose

Right. Not their intent, not what they were told they were going to be working on, and not something they were comfortable with.

michael barbaro

And Kevin, thinking about this, it’s hard to imagine that the U.S. military would not use U.S. technology, American-made technology like this, in warfare. That seems inevitable, doesn’t it?

kevin roose

Well, right. I mean as long as there has been a U.S. tech industry, there’s been a partnership with the U.S. government and with the military. I mean, the original internet came out of a Defense Department project.

michael barbaro

Mm-hmm.

kevin roose

And since then, lots of advances in technology have been spurred by this collaboration between the Pentagon and Silicon Valley. And a lot of these companies already provide technology to the military. I mean, a lot of military computers run Windows’s operating system. But this was a bridge too far for these employees. They say, we’re O.K. basically providing general-use technology, the same stuff that you and I could go and buy. But the specific development of our technology for use in this specific case crosses a line for us, and we don’t want to be a part of it.

michael barbaro

So Kevin, what’s interesting to you about this story?

kevin roose

Well, I’m interested in the ethical debate around the use of artificial intelligence and other technologies. But I’m also a business reporter. I think this is a decision that is not just being made along ethical lines. These companies are also thinking about their businesses and the amount of money and the profitability of developing these systems for the military. So I wanted to look at it along those lines, as well. Let’s just bracket the ethical discussion for a second and say, does this make financial sense?

michael barbaro

Well, not just bracket it, right? Isn’t a company often going to make decisions about moral dilemmas based on the implications to their finances?

kevin roose

Exactly. These are not philanthropies or think tanks. These are for-profit corporations that generally operate on a cost-benefit calculation. And so just on that plane, I wanted to see, does this make sense for them? [MUSIC] And so, I did a little bit of digging, and I went back about 50 years in history, and I found an example of a company that sold something to the military and later came to regret it.

michael barbaro

Which was what?

kevin roose

It’s the case of Dow Chemical. In 1965, Dow Chemical is this small, little-known chemicals company based in Michigan. They make lots of different things — household chemicals, agricultural chemicals.

archived recording

Now, at your favorite grocery store, the most amazing food wrap ever developed!

kevin roose

They were probably best known for making Saran Wrap.

michael barbaro

Heard of it.

archived recording (speaker 1)

Have you tried Saran Wrap? It really is amazing! And look, Saran Wrap clings like magic!

archived recording (speaker 2)

Saran Wrap is a product of the Dow Chemical Company.

archived recording (speaker 3)

[JET PLANES AND EXPLOSIONS] Vietnam. United States helicopter gunships backed up ground forces in a strong assault on a Viet Cong position —

kevin roose

Meanwhile, the Vietnam War is happening. And in 1965, Dow Chemical wins a contract with the Department of Defense to produce a new chemical called napalm B. And napalm, which is a highly flammable gel that binds and sticks to things and then burns them — had been in use. But napalm B was the new formulation, and the military was very excited in using this in Vietnam.

archived recording

Troops followed up the advantage gained by the air support to knock out the V.C.

kevin roose

They thought it had a lot of promise on the battlefield. It was hard to contain and hard to put out. It was a very effective form of weaponry — a very horrible form of weaponry. And it produced very, very bad burns on people. They would basically be burned alive. And so, there’s not much noise about this for the first little while. And then Americans start seeing images from the Vietnam War. And for a lot of people, this is their first time seeing the effects of napalm.

michael barbaro

Mm-hmm.

kevin roose

Most famously, there’s this photograph — you’ve probably seen it. It won the Pulitzer Prize. It is a photograph of a 9-year-old girl who’s been badly burned by a napalm weapon.

archived recording (martin luther king jr.)

So far we may have killed a million of them, mostly children.

kevin roose

And it sort of shocks the American public, and it galvanizes the anti-war movement.

archived recording (martin luther king jr.)

What do they think as we test out our latest weapons on them, just as the Germans tested out new medicine and new tortures in the concentration camps of Europe?

archived recording

[CROWDS CHANTING] No more napalm! No more napalm! No more napalm! No more napalm!

kevin roose

And they decide to go after the use of napalm by U.S. armed forces. And the way that they do that is by going after Dow Chemical.

archived recording

[CROWDS CHANTING]

archived recording (speaker 1)

This is Monday afternoon, November 20, on the San Jose State college campus. What you’re watching is the result of a demonstration that began about an hour ago.

kevin roose

They staged demonstrations outside the company’s headquarters, and they go to dozens of colleges where Dow was recruiting students to come work for them.

michael barbaro

Hm.

archived recording

The professors against the war and the Students for a Democratic Society had marched in front of the administration building to protest the recruitment on this campus by Dow Chemical Company of employees.

kevin roose

And they protest the recruiting sessions. They have signs like “Dow deforms children,” “Dow deals death.”

archived recording

They’re not allowing anyone in. Like, the only people they’re allowing in are those people that are connected with Dow. [PROTESTERS SHOUTING]

kevin roose

My favorite story from this period is that the Dow recruiters — the people that would go to campus to talk to the students — they would get locked up in classrooms. They would get, like, held hostage by the activists. And at one point, one of the recruiters got so used to this that he started bringing a sandwich in his briefcase, so that if the inevitable happened and he got locked in a room with a bunch of angry protesters —

michael barbaro

He would survive the day.

kevin roose

He would at least have lunch.

michael barbaro

So this sounds like a complete corporate nightmare — that your company is so associated with war and with death, and not with the household product that you make — Saran Wrap.

kevin roose

Right. And Dow was particularly offended by this because napalm was a very small part of its business. I mean, it accounted, I think, for a half of 1 percent of its annual sales. And yet it had sort of come to take over its corporate identity. So there was some internal discussion, there were some debates within the company and at the board level about what to do, whether to stop manufacturing napalm. And there were lots of reasons for that. I mean, there were customer boycotts. They did take a financial hit. And they had to do a lot to counter their corporate reputation as a warmonger. But what they really worried about was recruiting. They worried that on these visits to college campuses they weren’t going to get the best people. They weren’t going to get the scientists and the engineers who would help them in the years ahead. And as a result, they would have long-term costs, not just financial, but also in terms of what they were missing out on.

michael barbaro

The future employee with moral objections to napalm, who doesn’t come work at Dow Chemical and make the next blockbuster product.

kevin roose

Right. But ultimately, they decided to stand their ground. They said, we have a duty to produce napalm for the U.S. government. It’s our patriotic obligation, and we can withstand the controversy.

michael barbaro

Dow Chemical feels like such an extreme case, and it feels quite different from what’s going on with these tech companies today.

kevin roose

Yeah, absolutely. These are not totally comparable. I mean, the Vietnam War, at this point, was extremely unpopular. You had these horrific images coming back. There’s no good use for napalm, right? There’s no pro-social, healthy use for napalm. It’s a weapon. Whereas a lot of these technologies — A.I., facial recognition, image recognition, augmented reality — these can all be used for very productive and healthy things. It’s only when you put them into a certain context that they can be used for harm.

michael barbaro

And that would seem to be the case for most tech companies, right? They are developing products not explicitly for the military. They just happen to be used that way.

kevin roose

Exactly.

archived recording

Artificial intelligence, drones, warfare and Google — it’s a mixture that caused an uproar inside the tech giant, where the early motto was “Don’t be evil.”

kevin roose

There was a big controversy at Google last year, about this military contract known as Project Maven.

archived recording

Google is developing artificial intelligence to analyze drone video data.

kevin roose

And this was a Defense Department program that basically used A.I. and image-recognition technology to interpret video images.

michael barbaro

Mm-hmm.

kevin roose

So the same kind of A.I. that Google would use in Google Photos, or in its self-driving car unit to recognize images out on the road —

archived recording

It would apparently be used as part of a drone program.

kevin roose

— could be used to, for example, improve the targeting of drone strikes to make drone strikes more accurate, to be able to recognize certain people or certain buildings and be able to direct a drone at that target specifically.

michael barbaro

Again, not what Google engineers probably ever thought the technology would be used for.

kevin roose

Exactly, I mean, they didn’t sign up for that. And some of them didn’t even know that this was happening.

archived recording

Thousands of Google workers now protesting Google’s involvement in a Pentagon drone program.

kevin roose

So there was a big uproar. There was lots of heated internal meetings.

archived recording

At least 10, maybe 12 employees did resign.

kevin roose

Some resignations —

michael barbaro

Wow. People quit the company over it.

kevin roose

Yeah, people quit the company over it.

archived recording

Meantime, thousands of Google employees have signed a letter protesting the company’s participation in an artificial intelligence project by the Pentagon.

kevin roose

There was a petition that was signed by thousands of Google employees.

archived recording

Quote, “We believe that Google should not be in the business of war. Therefore we asked that Project Maven be canceled.”

kevin roose

Calling for the company to not only not enter into this contract, but to stop making tools of weaponry altogether. And ultimately —

archived recording

Tech giant Google will end a project with the Pentagon.

kevin roose

Google bowed to the pressure. They pulled the contract. They said they wouldn’t renew it. They put out a statement saying, we’ll still work with the military on other projects, but we’re not going to manufacture weapons and weapons-related technology. And that seemed to quiet the unrest. [MUSIC]

michael barbaro

I feel like people would be surprised that a huge tech company like Google would back down like that.

kevin roose

Yeah, I mean, it does speak to the difference between a conventional contractor like Dow Chemical and these tech companies. I think there are three main differences there. The first is that these companies are beloved. They have loyal followings and they don’t want to do anything to upset that. The second is that these companies are very idealistic. And in Silicon Valley, this is a big deal. Companies market themselves not just as profitable enterprises, but as humanitarian projects. And anything that compromises that hurts them.

michael barbaro

Right. World-changers.

kevin roose

Totally. And I think the biggest difference is that employees at these companies — especially the engineers who work on very specialized projects like A.I. development — have a ton of leverage, because there are not very many of them, they make a ton of money, and they’re really hard to recruit. If a couple of chemical engineers walk out of Dow Chemical in protest, they can probably find some more where that came from. But if hundreds or thousands of A.I. engineers are upset about a project that Google is working on, that’s a real threat to their business. It doesn’t take many people staging a protest, or threatening to walk out to really get the attention of senior leadership.

michael barbaro

So these three differences that you just laid out explain why Dow didn’t back down, despite images of dying children affected by their products, whereas Google did back down, long, long before anything like that happened to it.

kevin roose

Right. And I want to be clear: What happened at Google was the exception, not the rule. This is not happening in any vast quantity across the tech industry. At Microsoft, for example, they came right out and said, we appreciate the objections of this small group of employees. They can move elsewhere in the company if they want, but we’re committed to seeing this contract through. And they’ve defended themselves by saying, we have a patriotic duty to provide technology to the military. We think that we can advocate for the ethical use of technologies like AI if we’re at the table in these discussions. And they say that basically, these technologies could be good. These could save lives on the battlefield. The army has said it wants to use HoloLens not just for combat training, but for things like measuring vital signs, and monitoring soldiers for concussions, and having some sort of hearing protection on people’s ears. So those are the kinds of arguments they’re making to justify staying in these contracts.

michael barbaro

So the argument here being that engineers who came to these companies to do good, having their technology used by the military does not necessarily mean not doing good.

kevin roose

Yeah, I mean, that’s one of the arguments that they’re making, is that, basically, if the U.S. military doesn’t have the best technology available on the market, our adversaries will, and we will be endangered as a result.

michael barbaro

I guess the question is, do the engineers buy that argument?

kevin roose

Well, some of them probably do. But as we’ve seen, a number of them don’t. [MUSIC]

michael barbaro

So Kevin, we started by talking about this intersection of the moral and the financial as these tech companies navigate this question of, should they work with the military? What’s your understanding in Silicon Valley about how these companies are thinking about that?

kevin roose

So I think a lot of these companies have really shifted their view on this in the last couple of years. A lot of Silicon Valley companies used to just think of themselves as toolmakers, right? We build this stuff, and we put it out into the world, and people use it. And our responsibility, basically, stops at the moment of sale.

archived recording (mark zuckerberg)

To me, entrepreneurship is about creating change, not just creating companies.

kevin roose

And now, I think we’re seeing that these companies are being held responsible by the public and by their own employees, not just for the tools they’re building, but for how they’re being used in the world. And this is not just about the military.

archived recording

At this hour some 300 programmers are threatening to leave Microsoft unless the tech giant drops its contract with the U.S. Immigration and Customs Enforcement agency, known as ICE.

kevin roose

I mean, also, companies are being scrutinized for working with ICE or working with law enforcement.

archived recording

An Amazon worker is pushing the company to stop selling facial recognition technology to law enforcement.

kevin roose

Social media companies are being held responsible for bad things that happen on their platforms.

archived recording

Facebook defending Alex Jones this morning, saying the platform will continue to allow the video blogger to run video on its site, even as it claims to crack down on fake news.

kevin roose

This is really happening across technology. And as technology embeds itself into every part of our lives, we’re seeing that these companies are having to make new kinds of decisions. It’s not just a dollars-and-cents profit-and-loss calculation anymore. They also have to consider what might happen out in the world once they release these technologies.

michael barbaro

Which is another way of saying that the morality element of this is now playing a much bigger role than it has in the past, and perhaps is not beating out financial, but rivaling it?

kevin roose

It’s certainly part of the discussion now in a way it hasn’t been before. It’s not just about how much money we’ll make, or how many resources it will take. It’s about what we’re building, and why we’re building it, and who we’re selling it to. [MUSIC]

michael barbaro

Kevin, thank you very much.

kevin roose