Transcript

Robert Wiblin: Hi listeners, this is the 80,000 Hours Podcast, where each week we have an unusually in-depth conversation about one of the world’s most pressing problems and how you can use your career to solve it. I’m Rob Wiblin, Director of Research at 80,000 Hours.

We’re about the get some advice from someone who has 16 years experience in the White House, both about how to land a job in the US executive branch, and how to get more a lot done once you’re actually in there.

If you’re a US citizen and you are interested in moving to DC to work on AI policy issues, then we would potentially love to chat with you. In the show notes we’ll stick a link where you can apply for free careers advising from 80,000 Hours. And we’re especially keen to meet people with backgrounds in law, policy or computer science, so if that sounds like you do let us know about it!

Alright, here’s Tom Kalil.

Robert Wiblin: Today, I’m speaking with Tom Kalil. Tom Studied political science and international economics at the University of Wisconsin, Madison and worked on the Democratic presidential campaigns in 1988 and 1992, before serving in the White House under Obama and Clinton helping to design and launch national science and technology initiatives, from 2009 to 2017. That involved working at the White House Office of Science and Technology Policy First as deputy director for policy and then as deputy director for technology and innovation.

Robert Wiblin: He’s now a chief innovation officer at Schmidt Futures, a foundation, which among other things works to improve US science policy and identify and pursue 21st century moonshots. Thanks for coming on the podcast, Tom.

Tom Kalil: Happy to be here.

Robert Wiblin: So I hope over this interview to hear your thoughts of long term ism in government and draw out a whole bunch of your advice on how people can best pursue policy careers in the US and potentially in other countries as well.

Robert Wiblin: But first, what are you doing now and why do you think it’s really important work?

Tom Kalil: Well as you noted, I’m serving as the chief innovation officer for Schmidt Futures, which is a philanthropic organization founded by Eric and Wendy Schmidt. I believe in general, philanthropy has big opportunity to make investments in things that both the public and private sector are under investing in.

Robert Wiblin: Now, what kind of investments are those?

Tom Kalil: One of the things I’m interested in is that the federal government is serious about using science and technology to solve certain types of problems in areas like national security, energy, space, health, and basic science. But there are a lot of other areas where the relevant mission agency has little or no capacity to invest in science and technology. Those create systemic gaps in the country’s research and innovation portfolio. That’s an example of an area where a philanthropist or a foundation has more flexibility to address some of those gaps.

Robert Wiblin: Okay. So we’re not gonna focus too much on your current foundation work in this interview, because I’m especially interested to hear about your many years of work in government. I guess to help, to give people a sense of how they could potentially build their own career in policy and government, let’s just briskly walk through through your own career.

Tom Kalil: Sure.

Robert Wiblin: Yeah. What was the path you’ve taken to progressively having more impact or influence in the policy world from, I guess, say your student days to to now?

Tom Kalil: Yeah. My career has been a first or second order consequence of my decision to volunteer on the 1988 Dukakis for president campaign. I did that in the summer of 1987. I volunteered for a part of the campaign called the issues department. That is like bootcamp for would-be policy wonks, because you have to learn how to get up to speed quickly on a new issue, how to create a network of outside advisors. You learn this work ethic of staying up until it’s done, because you can’t ask a presidential candidate for an extension. You learn how to look for new policy ideas that your candidate can get behind, how to prepare them for presidential debates, how to understand what issues they’re likely to get asked about as they travel around the country.

Tom Kalil: You certainly don’t become an expert in one particular area, but you learn how to become a generalist, read lots of things, talk to lots of smart people and then try to figure out what does the candidate really need to know about this issue to be informed and knowledgeable.

Robert Wiblin: Presumably, that couldn’t have been the complete beginning, ’cause you can’t just walk off the street and get a job developing policy for presidential candidates. How do you actually get your foot in the door to take that kind of position?

Tom Kalil: It was really volunteering early. I was volunteering in the summer of 1987 when there were lots of different presidential candidates. There wasn’t the same level of competition that there might be. Then the other thing that I was lucky, is that the issue that I knew something about, namely international trade, was an area of disagreement between the candidates. The fact that I knew something about international trade made them more interested in bringing me on as a volunteer.

Robert Wiblin: How much do you think it ended up boosting your career that Dukakis actually became the nominee?

Tom Kalil: That was important. What was even more important is that a number of the people that I worked for also wound up working for Bill Clinton. In ’92, very close to the general election, in the fall of ’92 some of the people I’d worked with in ’88 contacted me and said, “Hey, do you want to come down to Little Rock and write some of Bill Clinton’s position papers?”

Tom Kalil: As you may remember, Bill Clinton won. The first thing that happened was that Bill Clinton had said during the ’92 campaign, “It’s the economy stupid.’ And he said, “In the same way that we have a national security council that will ensure that the president is focused on foreign policy and national security and defense issues every single day, we need to focus like a laser beam on the economy. So we should have a national economic council.”

Tom Kalil: So Bill Clinton asked Bob Rubin to be the first head of his National Economic Council. Bob Rubin had never worked in government before. He asked some of the campaign people, “Who should I hire?”

Tom Kalil: And they said, “Oh, you have to hire Tom Kalil.”

Tom Kalil: Bob was initially highly skeptical … fortunately I did not know that at the time. I thought that it was more or less a pro forma interview so I was very relaxed. I visited him in New York and he asked me why I wanted to work for the national economic council, what types of things would I work on? Fortunately I knew a lot more about technology policy than he did, since he was an investment banker. I came off as appearing very knowledgeable. He said, “You are as advertised,” and offered me the job. That’s how I wound up working in the White House in my late twenties.

Robert Wiblin: So is this a path that you think listeners could potentially take to get on board with a presidential campaign, early in it’s run, when it’s a bit less competitive to get involved? And someone who is in their mid twenties can potentially get a policy position?

Tom Kalil: Absolutely. Yeah. I think that it’s a career move that has a lot of variance. I don’t think you should do it if you say, “Well, I want to be guaranteed that my candidate will win my party’s nomination, win the general, and then I will get a great job.”

Tom Kalil: I think you need to be prepared for an outcome in which none of those things happen and still be okay with doing it. I think that there are a number of instances during my career where I’ve just been lucky. Things could have easily worked out another way and I would not have had the opportunity to work for two presidents for 16 years.

Robert Wiblin: Did you feel kind of out of your depth a bit when you first got involved in the campaign?

Tom Kalil: No, I don’t think so. I mean, I certainly felt a little bit more out of my depth. Maybe I was too young and brash to recognize it when I was actually doing it. But having the first time I worked in the government be working for the White House is a little unusual. So there was a fair amount of learning on the job that I wound up doing.

Robert Wiblin: How did you choose Dukakis? And I guess if you don’t want to choose a specific candidate, what guidelines could we use to figure out what campaign do I want to get involved with?

Tom Kalil: I chose Dukakis primarily because I was in the Boston area at the time. That’s where his campaign was located. It wasn’t based on a review of all of the candidates and then saying, “Which particular candidate is closest to my policy preferences?”

Robert Wiblin: Should people investigate the candidates a lot and think, “Oh, this is one that I agree with ideologically or should they maybe look at how likely they think they are to win? Or how likely they are to win the general?

Tom Kalil: Oh. So it depends, right? So I think that in a lot of cases, there are actually not huge differences between candidates of a given political party, and a lot of it is about how likely they are to win or what’s their personal story, what are their strengths and weaknesses as a candidate? For example, Mike Dukakis was very thoughtful. He had a number of weaknesses as a political candidate.

Robert Wiblin: Okay. So moving on, what were you working on in the Clinton administration?

Tom Kalil: Yeah, I worked for part of the White House called the National Economic Council, which was this new organization whose responsibility was to coordinate domestic and international economic policy. I handled the technology portfolio. The entire staff might fluctuate between 20 and 30 people. Everyone would have a fairly broad portfolio.

Tom Kalil: When I started, I spent a lot of time working on issues related to information and communications technologies. That was because Vice President Gore was particularly interested in what was then known as the information superhighway. But today we just call the Internet because back in 1993 the Vice President had this vision of this global network of networks that was going to connect computers and all these other devices. Exactly how it was going to evolve was not entirely clear. I did a lot of work on those issues. Then also towards the end of the administration, I worked on something called the National Nanotechnology Initiative.

Robert Wiblin: Yeah. So I imagine once you’re actually in the White House, things are so busy, there’s so much going on that it’s maybe hard to find time to think and to figure out what you actually believe and what the priorities should be. How do you choose areas to focus on? Do you just have advisors? Does everyone have advisors going down and down and down? People who actually have time to read original research and figure out what the most promising technologies are?

Tom Kalil: You’ve always got a situation in which you can have both formal and informal advisors. For example, within the White House, there’s an organization called the President’s Council of Advisors on Science and Technology. There’s an organization called the National Academy of Sciences. It has a different name now, but they’re continually producing reports. What you have is, first of all, some things that you have to deal with. For example, if a crisis like Ebola occurs, you don’t have the option of saying, “Well, this year I’d prefer not to work on Ebola.” When a crisis like that occurs, you have to work on it.

Tom Kalil: That’s one source of the agenda. Second is that presidents campaigned on things. They know that they’re going to be held accountable by the press and potentially the voters , “You said you were going to do X, Y, and Z. What sort of progress have you made?” There’s always this promises made, promises kept exercise.

Tom Kalil: There might be people who are bringing issues to your attention. I’ll give you a canonical example of this. Congress passed a law that said that if the executive branch wanted to change the definition of what constituted a supercomputer for purposes of export control laws, then the president had to make a determination that this was a sound policy. When Congress passed this law, they set it at a particular level in terms of the speed and power and performance of the computer. The problem was is that what is a super computer one day-

Robert Wiblin: Is a laptop the next.

Tom Kalil: Is a laptop. Apple would come to us and say, “Hey, this is going to be classified as a supercomputer. Do you think that makes sense?”

Tom Kalil: We would say, “No.”

Tom Kalil: There would be things like that that people would bring to our attention of saying, “Hey, public policies about to have this deleterious impact on US exports without any corresponding national security benefit. Don’t you think that you should change that policy?”

Tom Kalil: You’re right in the sense that a lot of times you have the intellectual capital that you walk in the door with and the amount of time that you have for reflection and, as the EA community would put it, cause prioritization is limited, because you are constantly dealing with fires.

Tom Kalil: Now I was a little fortunate in the sense that science, technology and innovation are by their very nature longer term issues. I worked on President Clinton’s Caltech speech in which he announced the National Nanotechnology Initiative. He talked about some of the goals of the effort. He said explicitly in that speech, “We may not reach some of these goals for 20 years. But that’s why there’s an important role for the federal government, because it’s very difficult for companies to justify making investments that may not pay off for 20 years.” Relative to someone else in another policy council, like the National Security Council that might be dealing with what’s happening in Bosnia that day, I had the luxury of occasionally taking some time to say, “What should I be working on next?”

Robert Wiblin: So after Clinton left, you went to UC Berkeley during the Bush administration, right?

Tom Kalil: Yes, that’s right.

Robert Wiblin: So again, this is a common cycle, I guess, the other side of politics is in is in power, so you’re in DC. Then when the other side is in power, people go on and do other things and build their careers outside of government. And then come back.

Tom Kalil: Yeah. I think that even if the party that you are affiliated with controls the White House, it’s not like people are going to stay there from the first day to the last day. What I did was somewhat unusual. So people burned out or decide they want to do something else. It’s not unusual for people to move in and out of government.

Robert Wiblin: What were you doing at UC Berkeley?

Tom Kalil: My job was to help the campus develop new multidisciplinary research initiatives that cut across different departments and colleges. So integrating biology, the physical sciences, engineering, computer science and social and behavioral sciences. One of the things that I worked on that might be of most interest to the EA community was a program called Big Ideas at Berkeley. The premise of the program was a lot of times students have ideas of their own and they just need small amounts of funding, a deadline, permission and tips about how to have influence without authority in order to help make their ideas happen. What I found to be very effective was issuing a very broad call for ideas and seeing what students would come up with, what they were intrinsically motivated to work on. Then, as I said, to give them funding connections to people on and off campus and tips about how to be effective in the world.

Robert Wiblin: Yeah. So influence without authority is one of these ideas that you are constantly returning to. Do you want to explain that one?

Tom Kalil: Sure. Yeah. Part of this stemmed from the roles that I would have in the White House, which is that I would have a desk and a phone and a computer and a business card that said the White House on it. But, particularly when I was working for President Clinton, it wasn’t like there were a bunch of people reporting to me. In order to be effective, not only would I have to come up with an idea, but I would need to figure out how to build a coalition around that idea, even though the people who I ultimately had to convince to say, “Yes,” in no way reported to me. The ability to do that, to build coalitions around ideas that you’re excited about, is very important in a policy role when it’s not like you have dedicated funding that you have direct responsibility for and a large staff that is reporting to you, that you have this command-and-control relationship with.

Robert Wiblin: Yeah. Is having influence but not authority mostly just a disadvantage because you can’t tell people what to do? Or does it have advantages as well, because maybe people are more open with you and less defensive?

Tom Kalil: I think the advantage is that the number of things I was able to work on was much higher. For example, if I had been someone who had a responsibility for a particular issue and had a budget and a staff, then people would look at me strangely if I said, “Well actually, I have an opinion about what you’re working on as well.” I think what I found intellectually engaging about the roles that I had is that I had a fairly broad remit. The name of my division was technology and innovation and we used to joke, “If it’s not technology, it’s innovation.”

Robert Wiblin: I guess another possible benefit is if you have lots of people reporting to you, then your whole time is taken up managing people and dealing with programs that are already running. Whereas you can think about new things and calling people up and just explore.

Tom Kalil: Absolutely. Yeah.

Robert Wiblin: Okay. So yeah. Ultimately you ended up at the White House Office of Science and Technology Policy for the full eight years from I guess 2009 to 2017. What were your key responsibilities there?

Tom Kalil: When I started, I thought that my primary role would be advising the person that President Obama had designated as his Science Advisor. This was Doctor John Holdren. I had known John because he was one of Clinton’s external science advisors, but he’d never worked in the government before. I was playing a role in the transition period. This is the period after the election has been held and before the inauguration has occurred. The Obama campaign called me up and said, “Would you run the transition team for the office of Science and Technology Policy?”

Tom Kalil: My job was to help whoever Obama was going to appoint to be a science advisor, get ready to do that job. So what are the issues that they’re going to have to deal with in the first hundred days? What are the commitments that Obama made during the campaign that they’re going to have a role in implementing, that type of thing.

Tom Kalil: So when Obama appointed Holdren to be a science advisor, Dr. Holdren asked me would I be willing to stay on as his deputy? I thought that my primary role would be, because this was John’s first time working in government as opposed to nearly advising government, that I would be able to give him some institutional knowledge about how to get things done. That’s how I sort of initially conceived of my job.

Tom Kalil: Then I asked John if I could start recruiting people to work on various projects that I thought were interesting. For example, during the Bush administration, I’d served as the chair for the Global Health Working group for the Clinton global initiative and gotten really interested in global health issues because of the high benefit-to-cost ratio of a number of global health interventions. I started recruiting people to work on things that I thought were important and then discovered that I really liked that, because I had the opportunity to recruit people in their twenties and thirties and just be able to work on a number of things in parallel. Then also, I enjoyed mentoring and coaching the next generation. I had discovered that as well. Then I really started building a team both to identify issues that I thought were important and then to try to find out someone who could work on those issues full time. Or occasionally just finding someone who I thought was really talented and giving them the opportunity to engage in public service.

Robert Wiblin: Okay. So maybe let’s dwell on this on this time for a bit. I mentioned that after someone gets elected they’ve got like 75 days or something until they actually take office. There’s going to be this very manic period when they’re trying to staff up quite a large executive operation.

Tom Kalil: Yes.

Robert Wiblin: I guess you were brought in soon after the election I suppose to figure out what are we going to do with the OSTP in this new administration?

Tom Kalil: Yes.

Robert Wiblin: I imagine that’s a time when if you can talk to the right people, there’s potentially a good chance of getting a job or a very unusually good chance of actually finding a role, so this might be a scramble, I guess, for people who are qualified to take positions that you’re thinking, “Oh well we want to talk about now in technology, want to have someone to do that.” Then you’ve got to very quickly find someone to do it.

Tom Kalil: Yes.

Robert Wiblin: Yeah. Do you have any comments on, I imagine people are probably trying to figure out how they’re going to staff the next administration in 2021 already, because you have to prepare so much.

Tom Kalil: Yeah, there’s actually not a very structured way of doing that. It tends to happen in earnest when the two parties have their nominees and then they can start working on their transition. Transition is usually sequencing and prioritization. What’s the order in which you want to try to get things done? Then staffing the administration. What you want to do is not only begin to identify candidates, but then you have to start vetting them, because particularly if they’re going to go through the process of Senate confirmation, there are all these reasons why people can fail to make it through that. So you’ve got to have these very difficult conversations with people to say, “If we put you through that process, is there anything that’s going to come out about you that’s going to embarrass you and the president elect.”

Robert Wiblin: So is there a bit of a phenomenon where if you’ve already served in a previous administration, then you’re viewed as safe? ‘Cause you’ve already-

Tom Kalil: If you’ve gone through that, yeah. If you’ve gone through the FBI background check or you’ve gone through … Senate confirmation is not necessarily for all positions. It’s necessary for a subset of those positions, particularly those that are presidentially appointed and Senate confirmed. But yeah, that’s a difficult process. Then what happens is that the administration will usually start with the most senior positions. For example, you need to have a Secretary of Defense right away. You can’t say, “Oh well, six months from now.”

Tom Kalil: It’s even to the point where in some cases, they might ask someone from a previous administration to stay. For example, the Trump administration asked our Deputy Secretary of Defense who was seen as not particularly political, he was a former colonel, a Colonel Bob Work, to stay on until they’d managed to get to identify and get confirmed their Deputy Secretary of Defense.

Robert Wiblin: Yeah. Do you have any advice for listeners who might be at a point in their career where they could imagine working in the next White House? What kind of things that they can do now to bring themselves to the attention of the right people? So they’re in a good position come November, 2020.

Tom Kalil: Obviously, it depends on what point in their career they are. If they are at the beginning of their career, then the question is, is there someone that can serve as an advocate for them? For example, if they’ve served as a research assistant for a professor who is advising people in government or has moved in and out of government before, they might say, “Oh, once we know who is filling this particular position, they’re going to be looking for a young, energetic, special assistant. That’s a role that you could play.”

Tom Kalil: Politics, all careers are this mix of know-how and know-who. I think that a lot of the reasons that you tend to pick people that you’ve worked with in the past, or people that you know, vouch for, is that there are a bunch of things about these roles that may be difficult to observe from the outside just by looking at your resume.

Tom Kalil: So is this the type of person who has a very strong work ethic? We used to call this the “Stay up until it’s done.” You can’t ask the President of the United States for an extension. The ability to produce a lot of work in a short period of time, because you might be in this situation where something has to be done by the end of the day. The ability to function without a lot of babysitting. Your boss tells you, generally, the problem that needs to be solved and you go figure out how to solve it. You’re not constantly coming back and saying, “Well what about this?” Right? Having a lot of common sense. Being someone that people like to work with. If you have this aversive personality and no one likes to work with you, that’s going to be a disaster in a political environment, whereas that might be less problematic if you’re an individual contributor as a software engineer.

Tom Kalil: These are characteristics and qualities of someone that you’re more likely to know if you’ve worked with them as opposed to looking at their resume. I think that there is this high level of reliance on recruiting people that you’ve worked with or people whose opinion you really trust, have worked closely with. Because it’s sort of like being in the foxhole with someone.

Robert Wiblin: Okay, we’ll come back to the career advice and what would make you a good fit later on, but what’s a big win that you’ve had at some point over your career? What kinds of things can people accomplish?

Tom Kalil: Yeah, I’ll give you a couple of examples. One was when I worked for President Obama, I was able to recruit at any one time, 20 people. Each of those people were working on a handful of things that were consequential. That wound up being pretty significant.

Tom Kalil: Let me give you one example. A young woman emailed me and the subject line of your email was, “Cass Sunstein says I should work for you.”

Robert Wiblin: That’s a strong subject line.

Tom Kalil: Good subject line. So I did a little research on her. It turned out that she had been a child violin prodigy with Itzhak Perlman, had won the major Yale undergraduate awards, was a Rhodes scholar, and was wrapping up a post-doc at Stanford in Decision Neuroscience. I went out on a limb and I decided to take a chance on her. Her name was Maya Shankar. I asked Maya, “What do you want to do?”

Tom Kalil: She said, “The UK has created this organization called the Behavioral Insights Team, which is taking these insights from people like Kahneman and Tversky and Sunstein and Thaler and using them to inform policies and programs. These are all US researchers. Why don’t we have something like this?” She said, “I would like to create that.”

Tom Kalil: Sure enough, in her late twenties, she arrived with no money, created this new organization called the Social and Behavioral Sciences Team, recruited 20 behavioral scientists to the federal government, got them to launch 60 collaborations with federal departments and agencies and got President Obama to sign an executive order institutionalizing this new entity.

Tom Kalil: I think that’s pretty consequential for someone in their late twenties to be able to accomplish. That’s one thing I did, was to recruit people of that caliber and teach them how to get things done in the federal government because the government doesn’t come with an operating manual.

Tom Kalil: I also launched a several dozen research initiatives. One you mentioned was the National Nanotechnology Initiative. That’s resulted in a $23 billion investment in nanoscale science and engineering. That has survived the transition from Clinton to Bush to Obama, to the current administration. In the Obama Administration, I worked on something called the Brain Initiative where the goal was to do for a neuroscience with the genome project had done for genetics. I was able to get DARPA prize authority, which they used for the self driving car competition. The second time they ran that a team won. Larry Page was at the finish line, he promptly acquired the winning team. That’s where the Google X division came from, was from that work on the self driving car. It’s also where Waymo came from. Then when I came back into government, I was able to get all agencies the authority to do incentive prizes for up to $50 million. If you’ve go to challenge.gov, you’ll see over 800 instances in which agencies have used this authority. Those are a couple of examples of the things that I’ve done.

Tom Kalil: But the thing that I’m most proud of, is the team that I built. Not only because of the impact that they had while they were there, but because I think many of them are going to return and engage in public service in the future. They learned some things about how to go from an idea to things happening in the world.

Robert Wiblin: So one of the biggest impacts was fostering young people to go off and have a great impact for their careers. It’s a little bit like 80,000 hours.

Tom Kalil: Yeah, absolutely.

Robert Wiblin: You get a lot of leverage potentially from that. Are there any products or businesses that you can point to that came out of, say, the Brain Initiative of an nano technology work? Like think of products that have reached fruition and had an impact?

Tom Kalil: Yeah. So the advances in Moore’s Law, the ability to double the number of transistors on an integrated circuit, increasingly rely on advances in nanoscale science and engineering. It’s having the biggest impact in areas like energy, particularly around energy storage, which is one of the big challenges that we need to address.

Tom Kalil: I have a story about the National Nanotechnology Initiative. In the late nineties I started interacting with scientists and engineers and said, “Well, if we decided to make this an area of focus, what are some of the things that might occur if we did that?”

Tom Kalil: They said things that were totally incomprehensible. They would say things like, “Well, we might be able to develop functionalized nano engineered MRI contrast agents. We might be able to develop a material where the Young’s modulus is of this many gigapascals. We might be-

Robert Wiblin: You were nodding along, pretending to understand.

Tom Kalil: Able to develop molecular electronics with a storage density of 10 to the 15th bits per cubic centimeter.”

Tom Kalil: What I did was I turned that into English, which was: we will be able to detect cancerous tumors before they’re visible to the human eye; store the equivalent of the Library of Congress in a device the size of a sugar cube; and make materials that are stronger than steel on a fraction of the weight. Armed with those examples, I was able to convince everyone, including ultimately the President, that we should make a bet in this area. For those people who are interested in science and technology, but don’t have a deeply technical background, one role that I’ve been able to play is just serving as an impedance match between the policy world and the scientific and technical world, because they’re generally not trained about how to communicate the importance of what they’re doing to a broader audience.

Robert Wiblin: Yeah. Okay. I’ll bring the other two questions I was gonna ask you. One is, is it just generally the case that science and tech people struggle to communicate with policymakers, ’cause they just thinking in a different way? And I guess, did you ever regret not having a more technical or scientific background yourself, having studied economics and political economy?

Tom Kalil: I have found the things, that there is a genuine value added that I can provide. Now the reason I’m able to do that is because I have a network of people who are deeply technical. Anytime I have a question like, “Is this a good idea or is this a crazy idea?” I have a network of people that I can talk to. But that there is a role that I can play, which is if I can understand it, then I’m generally capable of communicating it to someone who hasn’t spent … I’ve spent a good chunk of my career interacting with scientists and engineers. Most policymakers are not doing that. A value add that I can provide is the ability to talk to scientists and engineers, get the gist of what it is that they’re talking about, and then communicate that to a much broader policy audience.

Tom Kalil: Then the other thing is that- that there is a virtuous circle that kicks in with doing this. I’ll give you an example of this. In 1993, I was one of maybe a handful of people in the White House that was interacting with the computer networking research community. I was interacting with DARPA. DARPA had a very fast connection to the Internet that was say, a 100 to 200 times faster than a dial-up modem. I saw what Mosaic, which was the predecessor to Netscape, looked like, when you had a 45 megabit per second connection. It was immediately obvious to me that the ability to create this global information space that anyone could contribute to with all these hyperlinks and that anyone could use, was going to be a big deal.

Tom Kalil: The reason is that we were allowing the research community to live in the future. It was like going in a time machine and seeing what the world was going to be like 10 years later. Right?

Tom Kalil: And then, I could do the information arbitrage of going around and showing everyone in the White House what this was going to be like and why it was important. And so, for most people, the first time they heard about the Internet was from me. And so, because in 1993 it was like-

Robert Wiblin: It was not going to be on people’s radars as much.

Tom Kalil: It’s not on people’s radar screen. Unless you are a computer scientist or something like that, it’s probably not something that you were familiar with. So after that, when I said nanoscale science and engineering is going to be important, people are like, well, he was right about that Internet thing so maybe he’s right about this as well.

Robert Wiblin: I think it feels to me like grant makers and I guess anyone including me, whenever you hear about a new thing that you could fund in science and technology or something you can go and work on, there’s always a question is this bullshit or is this something that can happen. And I guess, how do you know who to trust? Because kind of even if someone’s trying to be very honest scientist, then it’s they’re so invested in their own thing that it’s very hard to know whether they’re hyping it beyond what’s technically practical.

Tom Kalil: I think that the important thing to do is to be taught to talk to more than one person. Right? So, for example, in late 2011 and 2012 when people brought me the idea that ultimately became the Brain Initiative, my initial reaction wasn’t: “great! Let’s announce this tomorrow”. It was to do due diligence on the idea by asking lots of other people, is it the right time to work on this? Right? Have various technologies from new materials to new types of imaging technologies advanced to the point where this is actually doable, or is this radically premature? And, if we were successful, how big a deal would this be? Is this a single or a home run. And then also, a lot of times, smart people disagree and you have to try to figure out what’s driving that disagreement. And then, if you’re the government, sometimes you might take a portfolio approach, right? So, if there’s a goal that you think is really important. As an individual researcher, you may care intensely about which path is the right way to do that because you have to stake your career on that. But, as a policy maker, you can be like-

Robert Wiblin: Give everyone a little bit of money.

Tom Kalil: Yeah, let’s support both and see which one works out.

Robert Wiblin: So, you’re like moderately technical, I guess. And, you’ve spoken to a lot of scientists and you’re trying to figure out whether something’s worth investing in. You speak to different experts and they maybe disagree. There’s a bit of controversy here. At that point you’re just like, well, maybe we’ll fund a bunch of different things, we’ll take a portfolio approach. Have you found any rules of thumb for figuring out who to trust more?

Tom Kalil: I think part of it is looking at someone’s track record. So, there are some people who they’ve just consistently delivered, number one. And number two, they do not have a track record of overhyping their work. So, if they say, I think that over the next five to ten years it will be possible to do X, you’ve just attach more of a weight to a statement like that than someone who has a track record of over hyping and under delivering. So, definitely someone’s track record. And then, the other is just ensuring that you’re talking to lots of people and not just one charismatic individual who may have an incentive to overhype their future research directions.

Tom Kalil: The other thing is also what’s the level at which you’re making a bet. So, for example, the idea of the National Nanotechnology Initiative, the bet that we were making was that at the nanoscale, new and potentially useful properties emerge. So, it’s like adding another dimension to the periodic table of elements. We did not say that carbon nanotubes is going to be the answer or buckyballs. We should put all our money in buckyballs. It was just there’s this new class of materials. They have new and potentially useful properties. If we can make nanoscale materials, devices, and structures, that’s going to have a broad range of applications.

Tom Kalil: So, similarly with the Brain Initiative, we didn’t say this is the way in which we’re going to figure out how the brain encodes and processes information. We were making the following bet informed by the research community, which is that if you look at the scientific revolution, what triggered it? It was advances in things like telescopes and microscopes that allowed us to see new things that we didn’t see before and allowed people to say, hey, Ptolemy and Aristotle, they got some things wrong. So really, if you look at say, what’s the driver of the scientific revolution? It’s the ability to see things that we couldn’t see before.

Tom Kalil: Similarly, the view in the research community was that neuroscience was tool limited. We can measure a very small number of neurons with high levels of temporal and spatial resolution, or we can take a fuzzy picture of your entire brain using something like fMRI, but we can’t do anything in the middle. And so, the hypothesis was: we could develop a set of tools that would allow us to ask and answer new questions about how the brain and encodes and processes information. So, part of it is also what’s something that’s big enough to matter, but it’s not so focused on a particular pathway that it’s the equivalent of going to Vegas and saying, I’m going to bet all my money on 12 and then if 12 doesn’t come up then I’ve lost all the money.

Robert Wiblin: Are there any things that you’ve helped to fund where in retrospect you think, oh damn, I should’ve realized that was not going to work or is that not how things function?

Tom Kalil: There were definitely things like that. So, for example, one of the things I learned is that if the United States is behind in a technology, it’s very difficult to try to re-establish a leadership position.

Robert Wiblin: Interesting.

Tom Kalil: Yeah.

Robert Wiblin: So, it’s better just to look at something else.

Tom Kalil: We tried to do that in the area of technologies like flat panel displays and we invested some money, but I don’t think a whole lot came out of it.

Robert Wiblin: So, is it like once there’s one city or one region that has really big agglomeration effects where everyone’s going to-

Tom Kalil: Once Korea and Japan dominated the market for things like active matrix, liquid crystal displays, then trying to get the United States back into that market is really, really hard, and might require more money [crosstalk 00:41:43] than the US is willing to put into it. Because obviously we believe that the primary role of government is to create the right environment for the private sector. It’s not to engage in this sort of heavy handed-top down industrial policy that you see a China engaging in, for example.

Robert Wiblin: So, I guess then you’d think, well, instead we should look at what’s the next thing going to be where we can potentially jump ahead?

Tom Kalil: Yeah. So, for example, we invested in this idea of flexible electronics where the idea is – maybe you have a display that’s a piece of paper that you can roll up and put into your pocket. And, if that’s an area where no one has established a clear leadership position, that’s more likely to be effective than saying, okay, we’re going to duke it out in some market that we’ve kind of already lost.

Robert Wiblin: You’ve already alluded to the fact that the administrations change every four or six years, well I guess occasionally it’s 12, and so you have to be thinking, well what can we do to potentially keep this policy around, to sustain it over a long term? What things do you do to make sure the kind of projects you make are built to last and how often do you succeed at that?

Tom Kalil: First question is you have to be sure that this is something that you really think needs to be sustained over a long term. So, for example, there might be some policies where you’re like, hey, there’s a really important role for the government for the first five to 10 years. But, that doesn’t mean that the government needs to be involved indefinitely. The technology might get to the point where it’s really time for the private sector to pick it up and run with it.

Tom Kalil: In the same way that the government began investing in the Internet in 1969 and eventually at some point the private sector, [crosstalk 00:43:14] beginning in the 1990’s, there was an explicit policy by the government of, hey, it’s time for the private sector to commercialize this and offer it as a commercial product or service. And now, we’re going to think about what’s after the Internet. So, we started at that point investing in something called the Next Generation Internet, which was an internet that was a thousand times faster than what was available through commercial products and services.

Tom Kalil: But, to your question, the thought experiment that you engage in is all of the people who are affiliated with the current administration walk out the door, and then what happens the day after? So, that’s the thought experiment that you have to do. And so, one of the things that you can do is you can ensure that there is a group of civil servants who genuinely thinks this is a good idea and will continue to work on this unless they’re explicitly told by the [crosstalk 00:44:10] incoming administration to stop.

Tom Kalil: So, for example, there have been a number of areas in environment and climate policy where the current administration has instructed the departments and agencies like EPA, I want you to reverse the Obama Administration’s policy on this, right? But, there are other areas like nanoscale science and engineering where in the absence of a direct instruction that all the agencies that are involved in this will continue [crosstalk 00:44:47] to work on it because they think it’s the right thing to do.

Tom Kalil: So, civil service buy-in, I think is really important, and it’s also very useful if Congress passes a law. So, there were some policies that we worked on, which they were done through regulation as opposed to through law. And, one of the things that makes those more fragile is if they can be done through regulation, they can also be reversed through regulation.

Robert Wiblin: Yeah. Legislation is much harder to change.

Tom Kalil: Legislation is much harder to change. So, for example, as I said, I was able to get a DARPA prize authority working for President Clinton and then every agency prize authority. And, agencies have continued to use this authority. Whereas if it had just been done through executive order, an incoming administration said, well we don’t like prizes, then they could just repeal the executive order. And, that’s a lot easier than changing the law.

Robert Wiblin: Yeah. Looking at the big picture of kind of US science and technology or innovation policy, what things could be changed that would allow the US to get significantly more R&D done, or to increase GDP growth or increase scientific advancement?

Tom Kalil: Yeah. So, as I said earlier, one of the things I’m really interested in is what problems do we have where science and technology could make a difference. So, right now our defacto answer is that we can use science and technology to advance the frontiers of human knowledge about ourselves and the world around us, to help with national security and intelligence, energy, space, and health. And, we have lots of other problems. And, for those problems, for one reason or another, we have not made a decision as a country, oh, let’s use science and technology to help solve that problem.

Tom Kalil: So, I think a really interesting thought experiment would be to take an agency that works on a really important problem, say K through 12 education. And say, what if that agency had the same capacity to mobilize the research community that DARPA did? Number one, what goals would it set? And number two, what are examples of projects that it would support in order to achieve those goals? So, right now in K through 12 education, productivity is negative. What I mean by that is we’ve doubled real per pupil expenditures without any corresponding significant improvement in student learning outcomes.

Tom Kalil: So, I believe that if we had a research agency that had the sort of same capability to mobilize the scientific and technical community that say a DARPA has, that there are things we could be doing to help address that problem. So, an example is DARPA supported this project and the goal was to reduce the time for a new Navy recruit to earn a technical skill and be able to do that in months rather than years. And, that’s developed some really dramatic results in terms of training. And so, you could imagine applying that approach to the training problem and to the education problem.

Tom Kalil: Similarly, a big problem that we have in the United States is the intergenerational transmission of poverty. So, we like to say that we have equality of opportunity, but we know that by the time kids show up at kindergarten, there are already large differences in school readiness, brain development, and vocabulary size. And, our schools do not narrow those divides. So, if you have low income parents, the odds of you struggling in school and struggling to make the transition from learning to read, to reading to learn, to graduating or going on to college. So, even though we talk about equality of opportunity, there are large differences in people’s life chances based on their parents, or what’s going on at home.

Tom Kalil: And, my view is not science and technology will solve all those problems. My view is if you are going to come up with a list of things to try, science and technology would be one of the things on that list. So, the thing that I’m really interested in is when we think about science and technology, by which I don’t just mean the natural sciences and engineering, I’m also talking about the social and behavioral sciences. What are the goals that we would like to use science and technology to help us achieve in the same way that we believe that we need science and technology to avoid technological surprise for the United States and to create it for our adversaries.

Robert Wiblin: You mentioned prizes before. I guess there’s different ways of trying to fund science. One is grants where you try to guess what’s going to work and you’ve got patents, I guess, as a market driven mechanism, and you’ve got prizes, which I think you’ve kind of been championing as perhaps something that the government should put more money into. Do you want to talk about that?

Tom Kalil: Yeah, absolutely. So, one of the things that I think is really perverse is that right now the federal government routinely will make a financial commitment that is contingent on failure. So, what do I mean by that? That is if you were to look at the finances of the United States government, you would find roughly $2.6 trillion in loan guarantees. So, that is the government saying if this individual or organization goes bankrupt, Uncle Sam as good for it. So, that is a financial commitment that is contingent on failure.

Tom Kalil: We are just barely scratching the surface on our ability to make financial commitments that are contingent on success. And, let me give you some examples of how when we do do that, it can have a large impact. So, there is a market failure associated with vaccines for diseases of the poor. Left to their own devices, drug companies won’t work on vaccines for poor people because poor people have no money. So Michael Kremer, I think you interviewed Michael’s wife-

Robert Wiblin: Oh yeah.

Tom Kalil: … on 80,000 Hours. So, Michael and Rachel wrote a book about this, but they came up with this idea for addressing this market failure, which is called an advanced market commitment. So, as a result of their work, five countries and the Gates foundation went to GSK and Pfizer and said, if you develop a vaccine for this disease, which is safe and effective, then we commit that we will buy X million doses at seven dollars a dose. And, that one intervention was enough to eliminate the 10 to 15 year gap that usually exists between when a vaccine is available in wealthy countries and when it’s available in low income countries. So, it’s a financial commitment that is contingent on success. So, it’s the Gates Foundation and five countries saying if you develop a vaccine which is safe and effective, then we’ll buy it.

Tom Kalil: So, the global health community has developed the capacity to do three things. Number one is identify a specific unmet need. The second is the ability to develop a performance based specification for what a successful innovation would look like. In this case, a vaccine that works and is safe. And the third is, if there’s a market failure, if there’s a huge gap between the social return and the private return to create some incentive that motivates the private sector to solve this problem.

Tom Kalil: So, my view is we should be doing more of this, right? Why is it the case that we have $2.6 trillion in financial commitments that are contingent on failure when we barely have any financial commitments that are contingent on success? So, that’s my view. And, these come in a variety of different flavors. There’s incentive prizes. So, that’s like the Ansari X prize saying if you develop a rocket that can go up a hundred kilometers, carry the equivalent of three people, repeat that within a two week period and do that without any backing from the government, then we’ll give you $10 million. So, that’s an incentive prize.

Tom Kalil: The advance market commitment or an advanced purchase commitment is if you develop X, then we will buy it. And then, there are other approaches like milestone payments. So, this is how the NASA collaboration with SpaceX worked. So, when we retired the Space Shuttle, the United States Government had to start spending a lot of money with the Russians in order to send astronauts to the International Space Station. So, they developed a collaboration with private sector companies like SpaceX, and they said, we want a rocket that will go up to the Space Station, deliver and retrieved cargo, and ultimately astronauts. And, we’re going to define a set of milestone payments that’s associated with getting there. And, every time you meet one of them, we will send you a check. And so, NASA got a capability on the order of $400 million that would have cost them two to four billion dollars using a business as usual approach. And, that’s what allowed Elon to finance the development of the Falcon 9 rocket. So, that was a win-win between the government and the private sector.

Robert Wiblin: Yeah, I think Alex Tabarrok, the economist has written a book, I think Creating the Innovation Renaissance, something like that. We’ll stick up a link to it where he talks about some of the pros and cons of prizes versus other funding mechanisms. Why hasn’t it taken off more? I guess one concern that I might have is that the government wouldn’t know how much it should offer or what’s an appropriate priced offer so it could end up accidentally giving away much more for something that’s too easy. And, maybe the political economy of it is bad because it gets taken over by businesses that are going to try to milk this system of prizes to get money for things they were going to do anyway. What kind of concerns do you think we will have that mean that prizes are just currently not a big funding mechanism for research?

Tom Kalil: Yeah. Well, certainly they’re not the appropriate way to support science in all instances. So, it works if you have a clear goal. So, if you know kind of the general direction that you want to go in, but you can’t articulate a clear finish line, then it makes a lot less sense to use a prize mechanism. So, in many cases, a researcher will know the question that they’re trying to answer, but they won’t really know what success looks like. So, that’s one thing is that is that it’s only appropriate for certain classes of problems where there’s a much clearer finish line. The ideal situation in the prize is, you almost don’t need judges, right?

Tom Kalil: So, if you think about the Ansari X prize, it was like, did the rocket go up a hundred kilometers? Was it carrying the equivalent of three people? Was it able to repeat that with an a two week period? So, you don’t need a panel of experts to decide, well did they actually do all those things? So, that’s sort of like the ideal case as opposed to-

Robert Wiblin: It’s more discretionary.

Tom Kalil: … something that’s more-

Robert Wiblin: Fuzzy.

Tom Kalil: … subjective. Right? It’s like, oh, is Rob asking an interesting question? Right? That we rely on peer review in order to be able to answer. So, it’s a lot more work upfront in order to do this as opposed to saying, well, I’m generally interested in ideas in this broad area as opposed to I can define what success looks like.

Robert Wiblin: I guess, is it possible that some research groups or businesses might find it hard to fund the money, if the money only comes later rather than beforehand.

Tom Kalil: Yeah. So, that’s why I’m interested in milestone payments because if someone had gone and said, okay, we want a rocket, and once you’ve developed it, then we’ll buy rides on the rocket. Particularly if it’s a new entrant, the fact that the government is saying, well I’ll buy it once you make it, might not allow them to be able to finance that.

Robert Wiblin: Are there any common themes of the ideas or the policies you’ve pursued that have succeeded versus the ones that haven’t panned out, or that failed later on?

Tom Kalil: That’s a good question. I don’t know that there is a single theme that connects all of them. I think that in terms of the role that I played, it is to bridge different worlds, right? So, there’s a policy world and there’s a science, technology, innovation, entrepreneurship world. And, by participating in both of those, I can help move ideas that are in one realm or community to another. So, that’s something that I’ve done over and over again. Or, when I found out about how the global health community was thinking about solving its market failures, I thought, this has broader relevance. It’s not just relevant to global health, it’s relevant to lots of other areas where there’s a market failure, and particularly when the innovation in question has high fixed costs and low marginal costs. So, part of it is just you encounter particular situations, and you discover that there’s sort of this underlying pattern language of approaches to solving a particular problem that can be applied in some other policy domain?

Robert Wiblin: It seems like a common thread of the other work that you’ve done and your writing is to focus on kind of these moonshots, these grand challenges, what are the things that are going to be really transform the world? Why focus on that rather than more incremental improvements. And, do you think, it’s important to have people work on incremental improvements, you just want to focus on the big picture?

Tom Kalil: Absolutely. Yeah. No, no, no. So, I mean, if you said, why has the world gotten better since 1820, it’s things getting better two percent per year, right? That has a huge impact over a long period of time. I think that part of it is that I think that setting these ambitious goals is important. And, I also think that it helps get people excited about investing in science and technology. Now you can take that too far, right? So, you can overpromise and underdeliver. So, I do think that there are some risks if you do that in an irresponsible way.

Tom Kalil: But, I have found it a way to get people excited about investing in science and technology more broadly or just focusing on particular problems. So, the UK for example, has done this report on the longterm economic consequences of living in a post penicillin world and it’s $100 trillion, right? So, I think people should be jumping up and down saying, hey, we should be doing more in this area. So, I do think that there are some areas where we’re underinvesting relative to the importance of the problem.

Tom Kalil: And so, I think that what you need to do is to go from … the experience of policy makers is they don’t have a shortage of problems. So, just showing up and saying, oh, and here are five more things for you to worry about. That’s not very effective. Right? And so, what you need to do is to say, here’s the problem and here’s what you could do about it. Right? So, what I find to be more effective is if I’m able to articulate both the problem and the solution because their experience is they have a list of problems as long as your arm, and just dumping another one on them, they’re like, gee, thanks. Well, what the hell do you want me to do about that?

Robert Wiblin: You said that what potentially the most valuable thing you were doing at the OSTP was recruiting the right people, was hiring. I suppose it’s feverish competition, obviously, over the best people. So, what could you do to get an edge in either figuring out who the best people were, where other people hadn’t realized how talented they were, or in convincing them to work at the White House where otherwise they wanted to do something else?

Tom Kalil: A lot of it was about me being able to be very specific and granular about what I had accomplished and what other similarly situated people had accomplished. So, obviously, I couldn’t compete on the basis of salary or the quality of our cappuccino machine. It was, hey, you’re going to be in the White House overseeing this federal government that allocates $4 trillion a year, and there’s no shortage of opportunities to make improvements relative to the status quo. And, here are all the tools that you have.

Tom Kalil: You can convince the President that this is an area that we should invest more money in. You can work with Congress to pass laws. You can identify things that federal departments and agencies can do through executive action, and you can leverage the President’s convening power to build coalitions, not only of government agencies, but of companies, research universities, philanthropists and foundations, nonprofit organizations, in order to achieve a particular goal. So, if there is something that you’re excited about doing, you will have access to tools in order to advance that agenda that you wouldn’t have anywhere else. So, it’s giving them, what some people call definite optimism, not just like general optimism of, oh, I think things will get better, of oh, there’s something concrete-

Robert Wiblin: Specific.

Tom Kalil: … that I can do to make things better.

Robert Wiblin: When people were skeptical, or kind of reluctant to come work for you, what were their reasons?

Tom Kalil: Well, I think that if your perception of the government is based on reading the press, what do you read about? You read about scandal, you read about gridlock, right? And, it’s not like those things are incorrect, but it would be like if your perception of New York were based on only reading the crime pages. I mean, say yes, those crimes do occur, but New York also has-

Robert Wiblin: Restaurants.

Tom Kalil: Restaurants.

Robert Wiblin: Parks.

Tom Kalil: Symphonies and amazing art museums. And so similarly, it is true that, particularly when you have divided government, it’s very difficult to pass legislation. And so, there’s a lot of things that we need to do that really require passing legislation. And, the fact that we’re not able to do that is a bad thing. So, I do not discount gridlock, partisan gridlock, as a problem. But, I have a different view that is based on, even subject to those constraints, things that I’ve been able to accomplish that I think are important. And so, I try to give that perspective as well.

Robert Wiblin: Yeah. So, do you think people underappreciate how much the executive branch can just do autonomously?

Tom Kalil: Yes. Yeah. Not only what it can do, but the President’s ability to convene.

Robert Wiblin: Explain that.

Tom Kalil: Sure. So, if the President invites you to a meeting, or even if someone who just works for the President like me, many people will show up. And, there are several different types of meetings that you could have. So, one would just be, hey, we think that this issue X is important. And so, we want to have a conversation about what are the best possible ideas to make progress on the issue X. So, that would be a way of trying to understand the issue, understanding potential courses of action. But, we might also have a meeting where the purpose is to serve as an artificial deadline. And, we would use the time between now and when the event was going to occur to ask people what it is that they might be prepared to do.

Tom Kalil: So, for example, the President said, in the same way that if you win the Superbowl or the NCAA, you get to come to the White House, the same thing should be true if you win a science fair, or robotics competition. So, he wanted to sort of increase the prestige and status of STEM education, and get more young boys and girls excited about STEM. And so, we would use that event, not just for the President to interact with these kids who are doing these amazing robotics competitions and science fair projects, but to mobilize the entire country to make specific commitments that would advance the ball.

Tom Kalil: So, the question that I would ask people is, if you were the President and you could call anyone and ask them to do something, who would you call and what would you ask them to do? And, sometimes there are organizations that are not part of the government, but are, particularly if they work together, are really in a position to move the ball forward on a particular national issue.

Robert Wiblin: Yeah. So, I guess I would have been skeptical as well that this power, at least with businesses, was very effective because you can imagine, you call in the CEO’s of various companies, and they come and play nice and they’re like, oh yes, we’re very interested in your priorities. Maybe because they’re scared of the government apart from anything else. But then, they go back and they do what they were going to do anyway because their responsibility is to the shareholders, and so they don’t have so much discretion to decide things based on what the President cares about.

Tom Kalil: Yeah. So, I think that if you asked them to do something that is contrary to the interests of their shareholders, you’re not going to have a lot of luck with that. But, for example, one of the issues that we were dealing with was the imbalance between demand and supply for technical workers, particularly jobs that you could get that didn’t require a four year degree, but you could get through some sort of accelerated training program. And, the status quo was within a given region, the companies were just stealing their workers from someone else. And so, we got them together and said, hey, what if as opposed to doing that, you grew the pie, and tried to figure out what sort of relationships you would need to have with various training providers so that there would be more workers with these skills as opposed to there’s this limited pool and what you’re doing is poaching someone else’s workers. Yeah, that makes sense. Right?

Tom Kalil: So, I think you’re right in the sense that if we said, we want you to take some action that is going to lower your stock price by 20 percent, they would have said no. Right?

Robert Wiblin: Go jump off a bridge, yeah.

Tom Kalil: Right, Or, hey, if you want to do that then you’re going to have to make that a law. Right? So, we don’t make it optional about whether or not you get to drive on the right hand side of the road. That’s just a law. And so, there are some instances where if you want people to do things, you have to pass a law. There are other instances where you can get people doing things by asking them, number one. And number two, putting it in the context of a broader social movement around a particular issue.

Robert Wiblin: And, do you think people do that because you’ve brought to their attention something that might’ve been sensible for them to do anyway, you just made it more salient? Or, perhaps is it they just had to buy into this vision and they’re like, actually this would be really cool, and this is something that this isn’t too costly, so I’ll happy for my business to be a part of it.

Tom Kalil: I think a part of it though is that you’re putting it in a broader context. So, a lot of times when we would talk to individuals, they would say, Tom, I feel like I am trying to put out a forest fire with an eyedropper. Right? So, I feel like my individual contribution relative to the scope of the problem doesn’t seem all that significant. But, if you were getting together all of the stakeholders, and we’re defining a set of mutually reinforcing steps that different sectors can take, then the piece that I’m able to contribute feels a lot more meaningful. So, that’s the thing that we can do. Because a lot of problems can’t be solved by an individual or a sector.

Tom Kalil: So, if we can say, here’s what government is going to do at the federal level, here’s what state and local government is going to do, here’s what foundations and nonprofits can do, here’s what the private sector can do, here’s what skilled volunteers can do, then everyone’s role feels more meaningful. Think about a campaign. The fact that you can only knock on a hundred doors, you’re like, well, what impact can that have? But, the reason it feels more meaningful is that-

Robert Wiblin: You know other people-

Tom Kalil: … you know other people are doing their part as well.

Robert Wiblin: You motivate people by coordinating them, by getting them all together and saying, well, if everyone else would do this, then it’s useful for me to do my part.

Tom Kalil: Yes.

Robert Wiblin: Yeah. Interesting. So, taking a step back to the hiring thing, were there any particular criteria you look for in hires when you’re interviewing people or looking at their [crosstalk 00:33:57]?

Tom Kalil: I was always more interested in people who had at least an initial theory of what they wanted to accomplish, that they were intrinsically motivated by. So, I put a lot of weight on intrinsic motivation because generally I find if someone really wants to achieve something, they’ll figure out how to do it. Whereas if they’re just responding to extrinsic motivation, they’re less likely to run through walls in order to achieve the goal.

Robert Wiblin: Interesting. So, typically people came in with a project that they were already interested in rather than arriving on day one and your being like, oh, what do I do?

Tom Kalil: Yeah, exactly.

Robert Wiblin: Interesting. So, I guess that means there’s pretty regular changeover on projects as people come in and out.

Tom Kalil: Yes, that’s right. Yeah. But, there were instances where I said I want to achieve X and therefore I want to find out the best person who can do that. So, for example, I knew I wanted to do more in incentive prizes. And so, after we passed legislation that gave every agency prize authority, I specifically looked for who can help lead a community in practice within the federal government so that more departments and agencies will actually start using this new authority. So, there were instances where I saw specific opportunity and then I mapped backwards from that, like, who could work out that.

Robert Wiblin: Is that a common situation, that if you want to get hired into government it’s important to have an area of passion, a policy that you really care about?

Tom Kalil: I think that’s helpful, yeah. I think that there are other instances where someone is just looking for a utility infielder. A lot of times someone will say, “I want someone who is really smart, hard working, has common sense, good people skills, the ability to pick up a new area quickly, the ability to write effectively”, so a set of skills that are useful in lots of different contexts. Then, they’ll just be able to catch whatever I throw at them and they don’t have to be an expert in a particular area.

Robert Wiblin: Are there other things you learned about hiring over 16 years in the White House so far?

Tom Kalil: I didn’t bat a thousand.

Robert Wiblin: Are there any indicators of people who didn’t work out?

Tom Kalil: Well, I guess some of it is learning to trust your instincts. If you have a small voice in your head saying, “I’m not sure about this”, it may be your intuition is trying to tell you something. That’s one thing, I think the other thing is, unfortunately sometimes you can really never tell until someone is in this environment the extent to which they’re gonna succeed or fail, so one of the things I benefited from is that a lot of these appointments were short term in nature and then if I really like the person I would work on getting it renewed, but the default was, they would time out after a year, or something like that. So that limited some of the downside associated with some of the decisions I made that I wish I hadn’t in retrospect.

Robert Wiblin: You often use this term, policy entrepreneurs and policy entrepreneurship.

Tom Kalil: Yes.

Robert Wiblin: Do you just want to explain why you think that’s an important concept and what that is?

Tom Kalil: Yes, I think a lot of times when people think about the government, maybe they have something in mind, like “Yes, Minister”, where the people in the government are trying to prevent anything from happening. Did you ever watch that show?

Robert Wiblin: I’ve watched all of it, yeah. I just say, looking at the UK at the moment, I think Yes, Minister would be a blessing if the government worked that well.

Tom Kalil: Right, right, right. No, but like, Sir Humphry is always trying to figure out how to maintain the status quo. Right?

Robert Wiblin: Exactly.

Tom Kalil: I do think that there are people whose … my main motivation for being in the government, if you had said, “Oh, your job is just to preside over the status quo”, I would’ve left after three months.

Robert Wiblin: Right, yeah.

Tom Kalil: That’s not … I wouldn’t find that intellectually engaging or worthwhile, so what I was always interested in was, what were the new ideas that were bubbling up from the research community or entrepreneurs that I could get behind and champion. The policy entrepreneur, in the same way that entrepreneur is identifying some unmet need in the marketplace and then trying to produce some product or service that addresses that unmet need through this process of opportunity recognition, the policy entrepreneur is identifying an opportunity for a new or improved policy that he or she thinks will advance the ball in some policy domain, and then is trying to figure out, what is the coalition that I would need to build in order to make that happen, and then actually going off and doing that.

Tom Kalil: So, you might have a couple different types of policy entrepreneurs. One is, someone who’s passionate about a particular issue. There was a woman in my staff, Jennifer Ericson, who was really passionate about reducing the waiting list for an organ transplant, kidneys in particular. She was super passionate about that issue and it’s be like midnight, or something like that, I’d say, “Jennifer, you gotta go home and get some sleep”. She was super fired up about working on that, so you have people like that, and then you have people like myself who are open to a pretty broad range of ideas. I view my value as, not that I’ve specialized in any one particular issue, but I’m curious about lots of different things, interested in lots of different things, therefore, like being in this situation where I’m getting a continuous flow of high quality ideas and then picking some of them and trying to figure out how many of them I can get to happen, as opposed to saying, “By the time I retire I want to have accomplished X”.

Robert Wiblin: Do you have any comments on the opposite of policy entrepreneurs? The Humphreys of the world who just want to kind of … there’s a role for those as well, people who try to stop things from being destroyed that are already working.

Tom Kalil: Well, yeah, so I think that it’s not always the case that change is a good thing. Right? I think that it’s important for there to be a productive relationship between political appointees like myself and civil servants because the civil servants: A- they’re the ones who implement things; B- they have a lot of institutional memory, so they can say, “Oh, we tried that before and here is what happened”. They’re also the ones who will be there when you leave, in many instances. Again, if you want a policy that persists, you want them to think that it’s a genuinely good idea, as opposed to something that we’re doing as long as you’re watching them.

Robert Wiblin: Yeah, that makes sense. What do you think people most misunderstand about policy entrepreneurship? One thing is that they just might not think of the policy world as being as dynamic as that.

Tom Kalil: Yeah.

Robert Wiblin: But what do they misunderstand about the process?

Tom Kalil: Well, I think a number of things. One is thinking that when the policy is announced the work is done, so one of the things I discovered is that that was maybe a third of the way through and that just because the president had signed an executive order or presidential memorandum, or Congress had passed a law, that you had to devote equal, if not more attention to implementation. I think people’s view was, “Well, if the President tells an agency to do something, of course they’re gonna do it.”

Tom Kalil: It’s like, –

Robert Wiblin: Maybe not.

Tom Kalil: Particularly as you are in the middle or towards the end of an administration an agency can rightly say-

Robert Wiblin: Maybe we’ll work this out.

Tom Kalil: Yeah, or just “Hey, the President has signed so many executive orders at this point that I can’t possibly do all of these and so therefore in the absence of some guidance I’m gonna figure out which are the ones that I think are the most doable.”

Robert Wiblin: What actually happens there? If the White House sends executive orders to agencies and they’re like, “We can’t do all these things, we just don’t have the resources to do everything we’re being asked”, do they prioritize themselves? Is there any recourse when it’s like, in a sense they’re not doing what they’re told to do?

Tom Kalil: Then what happens is that if the White House really cares, if the President really cares then there’ll be really frank discussions about some sort of down selection of which of these … it’s appropriate for an agency to say, “You’ve given me all these things to do, I can’t possibly do all of them.” I think you see that particularly towards the end of an administration. We were told, “Look, this particular part of the federal government, they’re gonna be focused on these three things and if you want them to do something else, “too bad”, or, “you’re gonna have to take it up with the Chief of Staff” because there’s been an agreement about what they’re gonna focus on and that means the only way for you to have a priority is to say you’re not gonna do certain other things. Like, if everything’s a priority, nothing’s a priority.

Robert Wiblin: How common would it be for there’d be a policy announced, it would go out to agencies and they would just kind of … it doesn’t really go anywhere ’cause there’s no one dedicated to it and you have to keep following up, calling every day?

Tom Kalil: Basically what you would see is a range of agency responses, some agencies taking really seriously and doing something meaningful, and other agencies sort of checking the box and engaging in the minimum level of compliance that they thought they could get away with.

Robert Wiblin: Is that ’cause they disagree about the policy sometimes, or just that they’re overwhelmed-

Tom Kalil: It could be that they’re overwhelmed, could be that they were never all that excited about the idea to begin with. Also, they might not have the capacity. For example, the President was more excited about STEM than his department of Education was so there was a large imbalance between the level of personal attention that he gave this issue and the extent to which the department was responsive to that.

Robert Wiblin: That means you kind of have to keep following up to-

Tom Kalil: Yeah.

Robert Wiblin: Keep reminding them.

Tom Kalil: Yes.

Robert Wiblin: Are there any other things that people misunderstand about policy entrepreneurship?

Tom Kalil: I think they believe that if you have divided government between the Congress and the Executive Branch then nothing gets done.

Robert Wiblin: Yeah, but actually lots of things going on.

Tom Kalil: Lots of things going on.

Robert Wiblin: Just without legislation being passed.

Tom Kalil: Yeah. The government is large and it has no shortage of legal authorities to do things and there are areas where it has significant amounts of discretion and it has the ability to convene coalitions of willing and able in order to achieve a particular goal.

Robert Wiblin: You’ve written this great article called, Policy Entrepreneurship at the White House, which is kind of lessons from your time in government. We’ll stick up a link to that so people can explore it in full if they’re interested, but briefly, what are one or two rules of thumb that you can give with working with big organizations?

Tom Kalil: One thing that I used to ask people is to imagine that you have a 15 minute meeting with the President in the Oval Office and he says, “Rob, if you give me a good idea for”, pick your cause, “reducing existential risk, then I will call anyone on the planet. It can be a conference call so there can be more than one person on the line. If it’s someone from inside the government, that I can direct them to something because I’m their boss, and if it’s someone outside the government then I can challenge them to do something. So, you not only have to tell me, what is your idea, but in order to make your idea happen, who would I call and what would I ask them to do?”

Tom Kalil: There are several reasons for this thought experiment. One is that if you work for the President you have the ability to send the President a decision memo and have him check the box that says yes. Over time that give you a sense of what psychologists call agency, a sense that many things that you see in the world around you are the result of human action or inaction, as opposed to the laws of physics. That’s one thing, a more expansive view of what do you think is potentially changeable. The second is, it’s sort of a version of the Hamming question, presumably if you really did have a meeting with the President you’d use it to describe an issue that you thought was really important as opposed to a secondary or third tier issue. The third is that many complex problems cannot be solved by a single individual organization, they require coalitions.

Tom Kalil: You can’t build a coalition if you can’t articulate, number one, who are the members of the coalition, and number two, what are the mutually reinforcing steps that you would want them to take. That’s one thing that I talk about in the Policy Entrepreneurship, then I also talk about something that people don’t ever really appreciate, which is that policy makers do things with words. What do I men by that? Well, think about when the priest says, “I now pronounce you man and wife”, he has changed the state of affairs by virtue of, A, him being a priest, and B, him saying, “I now pronounce you man and wife”. Similarly, the way that a policy maker both frames and makes a decision and implements that decision is through documents. When the President does it we call it an executive order or presidential memorandum, when a regulatory agency does it we call it a rule, when the Congress does it we call it legislation. But in all instances it is a document that you are creating or editing, so part of the policy process is that you are able to figure out what’s the document or documents that you need to create or edit and who is allowed to take that something from being a Word document that is on your screen to something that has some force in the world?

Tom Kalil: I would see this all the time, something would go from being a Word document on my computer to being a presidential executive order, it always seemed like this slightly magical transformation from a Word Doc to something that is instructing relevant members of the Cabinet to take some action.

Robert Wiblin: I guess this makes you more ambitious, then you’re like, “What is the best thing, what is the best memo-

Tom Kalil: Yeah, exactly.

Robert Wiblin: … that I could write.

Tom Kalil: Yeah, exactly, yeah. But also you have to be able to articulate some coherent relationship between ends and means. I would … a lot of times someone would come visit me and they would say, “my issue is important”.

Tom Kalil: I’d say, “great, let’s say that I’m prepared to stipulate that, what is that you want me to do?”, then they would look at me and they would say, “You should make this a priority”.

Tom Kalil: I’d say, “What would that look like?” People were not able … they were able to tell you that their issue was important and that they thought the President should devote more time and energy to it, but then when you said — “Alright, what is it, literally … let’s say we got the President super interested in this issue, what would they do?” They weren’t able to articulate that part.

Robert Wiblin: Yeah, I’m sympathetic to that ’cause there’s a lot of things that I think are very important, but I’m also not sure what should be done. I suppose maybe it makes sense for people to think more about that once they’ve gotten people to care about it, but at the same time, maybe it’s hard to get people to care about something if you have no actual concrete steps that they can take, they’re like, “Well, I don’t know what to do”.

Tom Kalil: Yes. Yeah, because that’s assuming that … because what you’re saying is, I’ve really thought about this issue a lot and I think it’s really important, but I don’t know what to do.”

Robert Wiblin: That’s a bad sign. So you should think about it.

Tom Kalil: Right?

Robert Wiblin: What were some of the aphorisms you had on the white board in the White House?

Tom Kalil: One was, the schedule is your friend. What that means is that things can take a while for things to happen in the government, to get done, but if you were able to land an event in which the President was gonna participate, then everyone had to meet your deadline. For example, if the President said, two weeks from now I’m gonna do an event on the brain initiative, then you’re allowed to go around and say, this needs to get done because it’s for the President. That’s what, the schedule is your friend means.

Tom Kalil: There’s an aphorism, talk to who owns the paper. What that means is that generally, as I said, if a executive order or a speech is being prepared it’s on someone’s computer so you want to find out who that person is. If you want to be-

Robert Wiblin: Where’s the file?

Tom Kalil: Right, if you want to be able to edit that or change that you need to talk to the person who ultimately has that document on their computer while it’s still … it’s still this sort of internal discussion about what it should say. People would always say things to me like, Kalil, how did you get the President to say X in the State of the Union?”

Tom Kalil: And I said, “Well, I talked to the speech writer.”

Robert Wiblin: I wrote the sentence, copy and paste.

Tom Kalil: It was probably like when Sherlock Holmes would explain how he’d figured something out and Watson was then disappointed, because once you say that it seems really prosaic. It’s like, “well, there’s this person, they write the speech, I talked to the person who wrote it, who was responsible for writing it and I told them to insert this line and he did it.” But unless you’ve been in the policy process you don’t really understand.

Robert Wiblin: I suppose a lot of people aren’t in a senior position to have a lot of those conversations. .

Tom Kalil: Yeah.

Robert Wiblin: Yeah, organizations. Another one that stood out to me is making it easy for people to help you is almost always a good idea. Do you want to expand on that one?

Tom Kalil: Yeah, people have many more things that they’re supposed to do than they have time to do, so if what you do is you show up and say, “I would like to give you another thing to do”, generally people are not gonna be super receptive to that. If there was something that, for example, I wanted my boss to do, my view was I should not show up and say, “I have a monkey that is on my back and I would like to transfer this monkey from my back to your back.” What I would figure out is, how could I make it as easy as possible for him to help me. For example, if I needed help getting a particular member of the Cabinet on board to support an idea that I was enthusiastic about, then I would say, “If I draft an email for you will you look at it, and if you’re comfortable with the substance and the content, will you send it to them?” He would generally say yes to that.

Tom Kalil: That’s what I mean, if you want someone to help you, make it as easy as possible. That also requires an understanding for an individual in the context of the particular organization, what’s easy and what’s hard? Right? You have to acquire a lot of fine grained institutional information about how different organizations work, about how decisions are made within that organization, and what’s relatively straightforward for them to do and what’s really difficult to do and what constraints they’re operating under.

Robert Wiblin: Yeah, the OSTP team has been famous for having something of a no credit ethos, something that you’ve talked about. Do you want to explain that? Was that something that just appeared ’cause of the kind of people that were there, or was it kind of an active effort?

Tom Kalil: I think it was an active effort. It wasn’t like, “Oh we’re allergic to ever getting credit for things”, it was, “if there’s a trade off between claiming credit and getting something done, always choose getting something done”. In other words, if someone else really cared about who got the credit, then we’d say, have at it.

Robert Wiblin: Donate it to them.

Tom Kalil: Right? If it’s like … there were situations in which a particular agency really wanted to be the person who got to speak immediately before the President and introduce the President. Right? So, if something symbolic like that is important to someone and it increases the extent to which they are excited about the idea, then let them do that.

Robert Wiblin: Did that extend to being inside the team, as well, that people didn’t try to jockey too much for who was responsible for doing things?

Tom Kalil: Yeah, absolutely. One of the members of my team, Doug Rand said it was a team of dolphins, not sharks.

Robert Wiblin: Is being able to trade away the credit for doing something unusually important in politics compared to other areas.

Tom Kalil: I think it’s probably generally useful, but I think in politics, given that there are some people who care, then I think it’s particularly useful to care less than they do.

Robert Wiblin: Did you ever worry that if OSTP isn’t getting seen as taking credit for things, then that would be bad for the agency ’cause then people wouldn’t care about-

Tom Kalil: No, part of it depends on who your audience is. I think that that by being a White House staffer as opposed to someone who’s going around and giving speeches, I had a lower public profile and so fewer people knew what it was that I had accomplished, but there was definitely an audience of people who are like, “oh, that’s the guy who actually did it.”

Robert Wiblin: Some people are known.

Tom Kalil: Right? It’s like your famous for 15 people, but as long as they’re the right 15 people then-

Robert Wiblin: So it’s kind of an apportionment of the credit, where it’s like to the general public the Secretary is responsible, but the people in the office know that it was you and so … yeah, okay, that’s interesting. What’s this bit it’s very important to be able to get excited about other people’s ideas? Do you want to explain that one?

Tom Kalil: Yeah. There are definitely ideas that I have come up with personally, but I think I have a higher than average ability to get excited about other people’s ideas. What that means is that the range of things that I can work on goes up considerably. I was not sitting at my desk and saying, “gee, I think we should launch a bit neuroscience initiative”, but the fact that a group of scientists and engineers contacted me and told me about this idea and I said, “Wow, this seems like a big idea” and was able to get excited about pushing it. For me, I didn’t need to be the person to originate the idea, I just had to be someone who was in a position to help it in order for me to get intrinsically motivated to work on it.

Robert Wiblin: From your experience in government, do too many want to originate ideas rather than follow them up?

Tom Kalil: Yeah, that’s a good question. Certainly within academia if you took the position that I did of going around and being excited about other people’s ideas then that wouldn’t be great for your career. People want to know what intellectual contribution have you made to the field, so someone being in my position of saying, “well, I didn’t come up with the idea for the brain initiative, but I played this important role in moving from an idea to things happening in the world”, that’s not something that you could really get away with in academia as a rule.

Robert Wiblin: Can you explain your magical laptop thought experiment?

Tom Kalil: It’s similar to the one that I talked about before which is, you have the meeting with the President. Instead, the thought experiment is that you have a magic laptop and the power of the laptop is any press release that you write will come true and what you have to do is to write a headline, which is a goal statement, several paragraphs that provide context, and paragraph level descriptions of who is agreeing to do what in the form, A does B, so C.

Tom Kalil: Again, the idea is, here we are in 2019, there’s some more desirable future that we’re working towards and we have to articulate who would need to do what in order to increase the chances of that desirable future coming true.

Robert Wiblin: I guess the benefit of this is it forces you to be very concrete about exactly what you want to happen?

Tom Kalil: Yeah.

Robert Wiblin: Where previously things are too abstract to … this is like, “oh, it should be a priority but it’s too vague to deal with”.

Tom Kalil: Yeah, it also gets you to think about what Bucky Fuller called the Trim Tab. What’s the Trim Tab? The Trim Tab is the thing that moves the rudder that moves the ocean liner. You can occasionally find instances in which there is an organization or an individual that is in a unique position to do a particular thing. I mentioned that I served as one of the chairs of the Clinton Global Initiative, one year Walmart announces that they were going to green their supply chain. Walmart has 180 thousand suppliers, or something like that, they’re all highly responsive when Walmart asks them to do something. Walmart greening their supply chain had far more of an impact than had they written a check to an environmental NGO. This A does B, so C really gets you to focus on the, what do you think in a particular problem or policy domain are the key leverage points and who would you need to act in order to take advantage of those leverage points.

Robert Wiblin: Yeah, when thinking about what actions you want people to take, how often did you get to think about actors outside of the United States?

Tom Kalil: It’s a good question, I did a little bit of that but not much. I was really much more domestically focused.

Robert Wiblin: Yeah, in effective altruism we’re often very interested in international coordination things. It seems like often existential risks and long term issues require us to look even beyond a national level. Do you have any thoughts on how you can get more coordination with the international level, or is that kind of outside of your bailiwick?

Tom Kalil: Well, one thing that I would observe is that individual personal relationships were still very important. Dr Holdren played a really important role in climate policy and one of the reasons that he was able to do that is there were a number of very senior people that he had worked with in the Chinese government that he had worked with when he was a Harvard professor with appointments at the Kennedy school and other research institutions. Those were relationships that have gone on for a long time and there was the reservoir of trust and mutual understanding that he could draw on when he was talking with his counterparts. Whereas, if these people are interacting with each other and they’ve never met before, that’s gonna be far more difficult.

Robert Wiblin: Do you think the one on one personal relationships are more important at the international level perhaps, ’cause we don’t know … their institutions and organizations and things going on in other countries are less scrutable to us so it’s all we can know is just this one person who we’ve met, whereas-

Tom Kalil: Yeah. No, I think so. Again, if the countries have diametrically opposing interests, then the fact that two people know each other … You know, you think about the negotiation with Iran, the two people who were involved with those negotiations at the technical level were both MIT PhDs, on both the US side, that was Ernie Moniz and his counterpart on the Iranian side. That doesn’t solve the game theoretic problems associated with the coordination-

Robert Wiblin: But at least they don’t hate one another, potentially.

Tom Kalil: It’s very … I don’t think you can overestimate the importance of people and having relationships that are based on trust, and mutual understanding and reciprocity. One of the reasons that I’ve been able to be effective is that there are a lot of people in the science and technology community that I’ve worked with on some specific project and if that experience went well for the other person when I call them up and say, “hey, let’s work on X”, I’ve got this sort of reservoir of goodwill that I’m drawing upon as opposed to, “who the hell is this guy?”.

Tom Kalil: There are these increasing returns that can kind of kick in at a certain point of your career where people say, “oh, Kalil is making this a priority. In the past when he’s done that, things have happened, therefore I should take it seriously.”

Robert Wiblin: It seems like this is a big debate the people have, how much that happens in governments or institutions is about the individual people and their personalities and their decisions and their relationships, versus it’s the institutional structure or it’s the incentives, it’s the gain theory, that kind of thing. I guess you want to say no, the people do really matter-

Tom Kalil: Yeah, people matter. I think unfortunately the answer to that is, both. There are definitely times when I have felt under constrained, so I felt like, I decide to work on this, I could have done otherwise. There were no institutional reasons that were saying, “oh, you should work on nanotechnology” I felt that I had autonomy, and I chose to work on it, not because … there was no political economy, there was no interest group story of saying, “Tom, you should work on nanotechnology”. It was just, I thought it was interesting and I decided to do it. As I said, because I’d been the person who showed everyone the internet in 1993 they were prepared to give me the benefit of the doubt.

Robert Wiblin: As you know, longtermism is kind of one of the distinctive things you find pretty frequently in the effective altruism community. How common is that long term style thinking or values in the policy world at the federal level?

Tom Kalil: I would say that there are pockets of it, but I think a lot of policy makers are focused on dealing with more immediate and shorter term issues. For example, imagine that you are President Obama and you’re coming in and the economy is in a free fall, you’re gonna focus on stopping the bleeding. If you’re involved in foreign policy and defense, a lot of your life is about dealing with immediate crisis. You might try to carve out some time to think about things that are important but not urgent, but it’s very difficult and requires a lot of discipline.

Robert Wiblin: Are there any departments or agencies or people who you mentioned who are more sympathetic to this world view, we can get things done?

Tom Kalil: Yeah. I think that in the science and technology area there is this notion that we’re investing now and it’s not gonna pay 