Transcript

Robert Wiblin: Hi listeners, this is the 80,000 Hours Podcast, the show about the world’s most pressing problems and how you can use your career to solve them. I’m Rob Wiblin, Director of Research at 80,000 Hours.

My guest today, Holden Karnofsky, is one of the most significant figures in the development of effective altruism. He has been applying his substantial intellectual energy to finding great giving opportunities since 2007, when he founded GiveWell with Elie Hassenfeld.

This has given him more hands-on experience of how to actually assess very varied giving opportunities, get around the huge uncertainties involved, and build a thriving research team, than anyone else I know.

As a result he has ended up as the primary advisor to Good Ventures, a cost-effectiveness focussed foundation which expects to give away billions of dollars over its lifetime.

I should also say that Good Ventures is 80,000 Hours’ largest funder, though I don’t think that’s changed anything about this interview.

We’ve recorded this episode now because the Open Philanthropy Project is hiring for a large number of roles, which we think would allow the right person to have a very large positive influence on the world. They’re looking for a large number of entry lever researchers to train up; 3 specialist researchers into potential risks from advanced artificial intelligence, as well as a Director of Operations, Operations Associate and General Counsel.

I’ll put up a link to their jobs page. If you’d like to get into global priorities research or foundation grantmaking, this is your moment to shine and actually put in an application.

This interview is likely to be of broad interest to most regular subscribers. However, if you have no interest in working at Open Phil, you can probably skip our discussion of their office culture and the jobs they’re trying to fill towards the end.

But don’t miss our discussion at the end of Holden’s public opinion survey of different utopias.

Without further ado, I bring you Holden Karnofsky.

Robert Wiblin: Today I’m speaking with Holden Karnofsky. Holden was a co-founder of the charity evaluator GiveWell, and is now the Executive Director of the Open Philanthropy Project. He graduated from Harvard in 2003 with a degree in social studies and spent the next several years in the hedge fund industry before founding GiveWell in 2007. Over the last four years, he has gradually moved to working full time at the Open Philanthropy Project, which is a collaboration with the foundation Good Ventures to find the highest impact grant opportunities. Thanks for coming on the podcast, Holden.

Holden Karnofsky: Thanks for having me.

Robert Wiblin: We plan to talk about some kind of thorny methodological questions that the Open Philanthropy Project faces, the kinds of people you’re looking to hire at the moment, because you do have a few vacancies, and how listeners can potentially prepare themselves to get a job working with you. But first off, while I think a lot of the audience will have some familiarity with GiveWell and the Open Philanthropy Project, maybe start by telling the story of what these organizations do and how they developed.

Holden Karnofsky: Sure. I’m currently the Executive Director of the Open Philanthropy Project, and I am co-founder, though I no longer work at, GiveWell, and they’re very related organizations, so maybe the easiest way for me to talk about what they do is just kind of go through the story of how GiveWell started and how that led to Open Philanthropy. GiveWell started in 2007 when Elie Hassenfeld and I worked at a hedge fund, and we wanted to give to charity, and we wanted to sort of get the best deal we could. We wanted to help the most people with the least money.

We found that we had a lot of trouble figuring out how to do this. We tried using existing charity rating systems. We tried talking to foundations, who largely didn’t tell us much. We tried talking to charities, who a lot of the time were kind of hostile and didn’t appreciate the inquiries. And at a certain point we kind of came to the conclusion that we felt there was not really a knowledge source out there that could help people like us figure out, between all the different things charities do, between, let’s say, if you’re trying to help people in Africa, providing clean water versus providing sanitation services versus providing bed nets to protect from malaria, which one could help the most people for the least money? And we were having trouble finding that, and we found ourselves very interested in it, and sort of more interested in it than our day jobs.

So we left the hedge fund, we raised startup funds from our former co-workers, and we started GiveWell. Today, GiveWell publishes research online that’s very detailed, very thorough, and sort of looks for charities that you can have confidence in. They’re cost-effective, they help a lot of people for a little money, they’re very evidence-based. The evidence has been reviewed incredibly thoroughly, and GiveWell also makes sure that there’s room for more funding, so each additional dollar you give will help more people.

GiveWell currently tracks the money that it moves to top charities, and it’s around $100 million a year is going to the charities that it recommends, on the basis of its recommendation. A few years after starting GiveWell, we met Cari Tuna and Dustin Moskovitz, and they were facing kind of a similar challenge to what Elie and I had originally faced, but very different in that they were looking to give away their money as well as possible, but instead of giving away a few thousand dollars a year like Elie and me, they were looking to give away billions of dollars a year over the course of their lifetimes.

So in some ways there was a lot of similarity between the challenge they were facing and the challenge we did. They wanted to give away money, they wanted to help the most people possible for the least money, and they were having a lot of trouble finding any guidance on how to do this, any research, any intellectual debate of any kind. Difference is that I think the task they were set up to do is fundamentally different from the task GiveWell is set up to do, and so we launched something that at the time was called GiveWell Labs and has since kind of morphed into the Open Philanthropy Project that is about trying to help people like that do the most good with the money that they’re giving away.

There are some really important differences. In some ways the two organizations have opposite philosophies, so GiveWell tends to look for things that are really proven, where the whole case can be really spelled out online. Open Philanthropy tends to look for things that are incredibly risky, often bold, often work on very long time horizons, and often things that we feel that no other funder is in a position to do and that it would a great deal of discussion and expertise and trust to understand the case for. And so they have kind of fundamentally different philosophies and fundamentally different audiences.

GiveWell now is, I think, functioning better than it ever has without me, and I am the Executive Director of Open Philanthropy, so what I currently do is I spend my time trying to build an organization and an operation that is going to be able to give away billions of dollars as well as possible. Our current giving is between 100 and $200 million a year, so we’re kind of at an early-ish phase in our process.

Robert Wiblin: If some listeners have only just heard about this idea of effective philanthropy, why should they care? Do you think it’s a particularly important issue that people should be focused on?

Holden Karnofsky: Yeah, I think it’s very important. I mean, I think some of the interesting thing about giving, whether it’s individuals giving to charity or large-scale donors giving to philanthropy, which is my current focus, I think one is that it’s just clearly an area where you can make a difference, where you can actually change the world. And I think when we look for the best things, the things that help people the most for the least money, I’ve been quite surprised personally by just how good they are and how much good you can do with how little money.

For example, GiveWell estimates that you can sort of avert an untimely death of an infant for every few thousand dollars spent, and Open Philanthropy kind of aspires, while taking more risk and working on longer time horizons, to do even better than that. So I think there are great opportunities to do good, and I also think it’s a very neglected topic. The funny thing is that there’s a lot of debate in our society about how the government should run, what the government should do. There’s even debate about what corporations should do, but there’s very little debate about how to do good charity, what foundations should do, what philanthropists should do.

It’s not considered an intellectual topic generally, and when people think about philanthropy, they kind of think of just a lot of warm fuzzy feelings, not a lot of scrutiny. They think about people putting their name on buildings, funding hospitals, and I think that’s really too bad, because actually I think when you look at the track record of philanthropy, it’s had some really enormous impacts. I think it’s changed the world for the better and for the worse and sort of been behind some of the biggest stories of the last century. For example, both the pill, the common oral contraceptive, and the Green Revolution, which is the set of developments that arguably led to several major countries developing and a billion people avoiding starvation, both of those had a really significant role for philanthropy, arguably were kind of primarily backed by philanthropy.

So philanthropy really can change the world, and in many ways I think it presents a given person with a greater opportunity to change the world than some of the topics that get more attention. And so I think the fact that it is so neglected and that it’s generally not considered an intellectual topic creates a huge opportunity to do what other people won’t and to have an outsized impact on the world.

Robert Wiblin: Initially GiveWell looked at a couple of different problems that people could donate to solve, including poverty in the United States, poverty overseas and, I guess, education in the US, and maybe some others, but it ended up basically focusing on helping poverty in the developing world, I guess especially extreme poverty and major health problems. Is that right?

Holden Karnofsky: Yeah, that’s right.

Robert Wiblin: What kind of problems does Open Phil now focus on?

Holden Karnofsky: Open Philanthropy has looked for causes that are what we call important, neglected and tractable, and so we have kind of a broad set of causes, and especially because when we were getting started, we wanted to try a few different things and get a feel for a few different kinds of philanthropy. One thing that we don’t currently work on is global health and development, and the reason for that is that we think GiveWell’s top charities are very strong options if you’re looking to directly help the global poor. We aren’t sure. Maybe if we put in a lot of work and a lot of research, we would find things that are higher risk and better, but it’s not at all obvious that we would, and we feel that GiveWell presents an outstanding option and something that’s very hard to beat in that domain.

And so what Open Philanthropy has tried to do is look for other ways, other schools of thought on how you might have a lot of impact. The basic story here is, if you start from the place of I want to help an inordinate number of people or do an inordinate amount of good with my money, what are some ways that that might happen? The sort of GiveWell approach or the GiveWell philosophy largely comes down to the idea that by sending your money to the poorest parts of the world, to the people who need it most, you can have more impact than by sending it, let’s say, to your local community.

Open Philanthropy works with some different theories. We work on US policy, with the theory being that as Americans, as people embedded in the US, we do have some understanding and some networks of the US policy landscape, and there is a decent historical track record where sometimes relatively small expenditures by a philanthropist can have a big impact on how governments make decisions, can help governments do better work. And so you can get a big, in some sense, leverage there. You can get a big multiplier.

Another thing that we’re very interested in is scientific research, so that’s another case where, by spending a relatively small amount, you can be part of developing a new innovation that then becomes shared for free, infinitely and globally, and you can have an outsized impact that way. And then a final category for us is global catastrophic risks, where we feel that the more interconnected the world becomes, the more it become the case that sort of a worst case scenario could have a really outsized global impact, and there’s no particular actor that really has the incentive to care about that. Governments do not have the incentive, corporations do not have the incentive to worry about really low-likelihood, super duper worst case outcomes.

So those are some of the areas we work in, and specifically the causes that we work in have come from a mix. We spent a lot of time, we spent really a couple years, just trying to pick causes and doing research on what’s important, neglected, and tractable, and we picked our causes through a mix of what our research found and also, frankly, through our hiring. One of our philosophies is that a lot of how much good you do has to do with who you hire, whether you have the right person for the job. A lot of our philosophy is about finding great people and empowering them to be creative and make individual decisions and have autonomy.

And so a lot of the causes we work in have been shaped by whether we’ve been able to find the right person to work in them. So through that we’ve come to a relatively small set of major focus areas, and then another set of causes that I won’t go into here. The major focus areas we have at this moment, we have a big investment in criminal justice reform. We feel that the US over-incarcerates people really badly. We feel that thorough research has made us conclude that this is not helping basically anyone. This does not provide a public safety benefit, that we could reduce incarceration, reduce the horrible cost of prison, both human and financial, and really maintain or improve public safety.

And we work in that cause partly too because we think there’s an opportunity to win. We think it’s politically tractable. So out of all the causes of comparable importance, many of them you’re never going to get anything done unless you can get something done at the national level, but criminal justice reform is a state and local issue, and we think it also is sometimes less of a partisan issue. A lot of times people on the left and the right are both interested in reducing prison populations, because that is cutting government, that is saving money, and that is often helping the least privileged people. So that’s a major cause that we work in.

Another major cause we work in is farm animal welfare. We believe that one of the ongoing horrors of modern civilization is the way that animals are treated on factory farms, just incredible numbers of animals treated incredibly poorly on factory farms. We believe that we’ve seen opportunities that are relatively cost-effective where our funding has helped contribute to a movement of getting animal welfare to be a higher priority for fast food companies, for grocers, and we believe that this has led to incredible amounts of just reduction in animal suffering, improvement in farm animal welfare, for relatively small amounts of money and good cost effectiveness. So that’s a major area.

And then two other major areas are global catastrophic risks, so we work on biosecurity and pandemic preparedness. If you were to tell me that somehow the human race was going to be wiped out in the next 50 years and ask me to guess how, one of my top guesses would be a major pandemic, especially if synthetic biology develops and the kinds of pandemics that are possible become even worse, and the kind of accidents that can happen become even worse. So we have a program that’s really focused on trying to identify the worst case pandemic risks and prepare the global system for them so they become less threatening.

And then another global kind of catastrophic risk we’re interested in is potential risk for advanced artificial intelligence, which I think is something you’ve probably covered before in your podcast-

Robert Wiblin: Had a couple episodes about that.

Holden Karnofsky: … and happy to get into more detail on it. Okay, cool, and happy to get more into that in a bit. Finally, we do a fair amount of work to support the effective altruism community itself, including 80,000 Hours. And then we have a whole bunch of other work. I mean, we do climate change, and we work on some other US policy issues.

Finally, another major area for us is scientific research funding. That’s where we kind of look for moonshot science that we can support, for example, trying to speed up the use of gene drives technology to eradicate malaria.

Robert Wiblin: So on the global catastrophic risks and science research sides of things, we’ve had an episode with Nick Beckstead, who works at the Open Philanthropy Project, where we talked quite a lot about those two. And on the factory farming, we have a very popular episode with Lewis Bollard, where we spent about three hours talking about all of the different angles on that. So if you’re interested in hearing about those two causes in particular, then we have other episodes for you to check out that I’ll stick up links to. And I’ll also mention that one of the last things we’re going to talk about is a whole lot of vacancies that are coming up at Open Phil, so this interview could be quite long, but if you’re interested in working with Holden, then stick around or maybe just skip to the end to hear that section about what kinds of people they’re looking for and how you can apply to work at Open Phil.

Let’s just dig in deeper and find out a bit more about how Open Phil actually works. How much are you hoping to dispense in grants over the next 10 years?

Holden Karnofsky: Well, over the next 10 years, we’re not sure yet. I mean, what we know is that Cari and Dustin are looking to give away the vast majority of their wealth within their lifetimes. We also believe that we want philanthropy to become a more intellectual topic than it is, and we have kind of aspirations that if we do a good job, if we have useful insights on how to do great philanthropy and help a lot of people, that we will influence other major philanthropists as well, and we’re already starting to see small amounts of that.

So over the long run, I know I’m looking there. Over the next 10 years, I think it’s really TBD, and I think we want to just take things as they come. At this stage, we’re giving away between 100 and $200 million a year, and our priority right now is really to learn from what we’re doing, to assess our own impact over time, to build better intellectual frameworks for cause prioritization and deciding how much money goes into which cause and which budget. And so I think we have a lot of work to do just improving the giving we’re already doing, and I think it’ll be better to wait until we have a really strong sense of what we’re doing and what we’re about before we ramp up much more from here. So the next 10 years specifically, I think that just will depend on how we’re evolving and what opportunities we see.

Robert Wiblin: How do you make specific grant decisions? What’s the process by which you end up dispensing money?

Holden Karnofsky: The basic process, the basic story or the kind of pieces of philanthropy as I see them, the job of a philanthropist, is first to pick causes, pick focus areas, pick which issues you’re going to work on. So that would be like, for example, saying we’re going to work on criminal justice reform, we’re going to work on farm animal welfare, and that’s something that, as I mentioned, and I think somewhat different from many other foundations, that was something that we put a couple years into by itself. The next thing that a philanthropist needs to do is basically build the right team. And so those are two things that I think are very core to Open Philanthropy, and two things that make us what we are, and to the extent we’re doing a good job or a bad job, it comes down to which causes we’ve picked and which people we’ve hired.

When it comes to the grant-making process, what happens is we generally have a program officer who is the point person for their cause. For criminal justice reform, it’s Chloe Cockburn. For farm animal welfare, it’s Lewis Bollard. And that person really leads the way. One of things we’ve tried to do at Open Philanthropy is maximize the extent to which this can be a very autonomy-friendly, creativity-friendly environment. We don’t slap any rubric or requirements on what kind of grants we’re going to do, and we try to minimize the number of veto points, the number of decision-makers. So it’s kind of one person out there talking to everyone, doing everything they can to educate themselves, and bringing the ideas to us. And then once a program officer wants to make a grant, we have a process, which you can link to, we’ve written about it.

Basically, they complete an internal write-up, which asks them to answer a bunch of questions like what is special about this grant, how will we know how it’s going, when will we know how it’s going, what predictions? We ask people to make sort of quantified probabilistic predictions about grants. What are the reservations? What’s the best case against this grant? That person writes up their case and then it gets reviewed by me and by Cari Tuna, who’s the President of Open Philanthropy, and that leads to the approval. And then we get to the part of the grant process where we need to basically get the payment made and get all the terms hammered out. So that’s kind of the general outline.

Robert Wiblin: Let’s look at this another way. Let’s imagine that I was a billionaire and I was looking to give away quite a lot of my wealth, but I didn’t really know where to start. What would you suggest that I do?

Holden Karnofsky: I would say that when you’re starting a philanthropy, the kind of first order questions are what areas you’re going to work on, what causes, and then also who you’re going to hire, what kind of staff you’re going to build, because you’re not going to be able to make all the decisions yourself when you’re giving away that kind of money. I feel like really the most important decision you’re going to make is who is making the decisions and who is making the grant proposals and how are they doing that?

What I would urge someone to do if they were a billionaire and they were just getting started, if I had one piece of advice, it would be to work really hard at those two decisions and take them slowly. I think that a lot of times when you go around asking for advice in philanthropy … and I think this kind of reflects that philanthropy is not considered such an intellectual thing … a lot of experienced foundations will tell you, you should really just start with the causes that you’re personally passionate about. You should take the things you’re personally interested in. If homelessness in your home town, for whatever reason, if that’s what strikes you, then that’s your cause, and that’s what you’re going to work on.

And then from there, you should maybe hire people that you already know and already trust, and then start figuring out what your processes are going to be. Open Philanthropy does … it’s not how every foundation thinks about it, but it’s common advice … and Open Philanthropy does actually sort of take the opposite view there. We think that picking a cause, if you’re going to work on homelessness in your home town, versus maybe criminal justice on a national level, versus maybe potential risk for advanced AI, that sort of makes maybe like a huge amount of the difference, maybe almost approximately all of the difference, in how much good you’re ultimately going to accomplish.

And furthermore, some causes are kind of naturally popular, because they’re just naturally appealing. Cities that have a lot of wealthy people are going to be cities that have a lot of philanthropy going into them. Causes that are kind of easy to understand and immediately emotionally resonant are going to have a lot of money going into them. And so oftentimes it’s the causes that take more work, more thought, more analysis to see the value of that may actually be your best chance to do a huge amount of good.

As a billionaire starting off, I would say your first job is to pick causes and probably then to pick people, and my biggest advice is to do it carefully and to take your time. I think I understand the advice to pick causes quickly, because it could be very daunting and frustrating to not have causes, and it’s much easier to get stuff done and to have a clear framework when you do have causes. And certainly we spent a couple years trying to pick causes, and during that time it was just a strange situation. It kind of felt like we were doing not much, but in retrospect I’m glad we put in all that time and maybe even wish we’d put in more.

Robert Wiblin: So how did you go about picking those causes?

Holden Karnofsky: When we were originally picking our causes, we basically had these three criteria: importance, neglectedness, and tractability. Importance means we want to work on a cause where a kind of win that we could imagine, or an impact that we could imagine, would be really huge and would benefit a lot of persons and would benefit them a lot. Then there’s neglectedness. So on the flip side of this, a lot of the most important causes already have a lot of money in them. A lot of what we did is we tried to figure out where would we fit in and what would we be able to do differently?

And then there’s tractability. So all else equal, we’d rather work in a cause where it looks more realistic to get a win on a realistic timeframe. And the way that we literally did this is we did these sort of shallow investigations, then medium investigations. We had a large list of causes. To give you an example, we kind of wrote down all of the global catastrophic risks that we had heard about or thought about, things that could really derail civilization, everything from asteroids to climate change to pandemics to geomagnetic storms. And then for each one we kind of had a couple conversations with experts, we read a couple papers, and we got an initial sense, how likely is this to cause a global catastrophe, what are some of the arguments, who else works on it, where is the space for us, and what could we do to reduce the risk?

We had this kind of not super-quantitative but definitely systematic, and we’ve got the spreadsheets on our website, where we rate things by importance, neglectedness, and tractability, and took the things that stood out on those criteria. And then we started looking for people to hire in them, and then the hiring process somewhat further determined what causes we really got into.

Robert Wiblin: I know that when you chose the particular problems that you did, you kind of committed to stick with those for quite a number of years, because you thought you had to spend some time to really develop expertise to even know what to do. Do you think that you got the right answers in the first place? Are there any things that you would do differently if you were doing it again today?

Holden Karnofsky: I think we have learned a lot since then, and I think we’ve had a lot of updates since then, so for example, I’ve written publicly about a series of major opinion changes that I had that all kind of coincided. This blog post called, “Three Key Things I’ve Changed My Mind About”, where I currently have a higher estimate of the importance of potential risk from advanced AI specifically than I used to, and that also kind of made me update in the direction of a lot of the causes that some of the most dedicated effective altruists have kind of done a lot of their own research and recommend. I think I moved in the direction of a lot of those causes.

I certainly think we’ve updating our thinking, and I’m glad we got moving in a reasonable period of time. I’m glad we got some experience. I’m glad we got to try different things. And I think the causes right now are really excellent. The ones that we’ve scaled up, we scaled them up over a period of years, and so we were able to see them as we were doing it. And so I think they’re excellent causes, but I don’t think that that intellectual journey is over, and I think in some ways we’re at the beginning of it. Sure, we picked some initial causes and we made some initial grants, and we learned some things about how to give, but our next big mission is trying to figure out which causes, which focus areas are going to grow the most.

And as we ramp up giving, what the relative sizes of the budget should be, what the priorities should be, how much money goes into each thing, and I think you could see our effective priorities, they may shift over time as we continue to tackle that question, which I think is a very thorny, somewhat daunting question. I could even imagine it taking decades to really work out, but of course, we’re going to try and make pragmatic amounts of progress, to make pragmatic amounts of increases in our giving.

Robert Wiblin: Do you expect to still be here in a few decades? I don’t mean you, but will Open Phil and Good Ventures still exist in a few decades, or is the plan to gradually wind them down?

Holden Karnofsky: Cari and Dustin want to give away the vast majority of their wealth within their lifetimes. That’s what they’re looking for. I could imagine there being a case to do it faster than that. You know, it depends on how things play out and how we think today’s opportunities compare to tomorrow. I think it’s very unlikely that it will be slower. In other words, I don’t think they have interest in leaving behind an endowment.

Open Philanthropy is different. I see Open Philanthropy as sort of an intellectual hub for effective philanthropy, almost could serve some of the role of a think tank, although it also operates as a funder. And so I think that if at some future date, Cari and Dustin have spent down all their capital, I’m guessing and I’m sort of hoping, I guess, that if it’s doing a good job, that Open Philanthropy is still around, helping other philanthropists make the most of their money.

So I see Open Philanthropy as more of an intellectual institution that has no particular reason to go away at any particular time, as long as it’s doing good work. Obviously, it shouldn’t continue to exist if it’s not, whereas Cari and Dustin, they have a fortune and they’re looking to give it away in a certain period.

Robert Wiblin: Could you see Open Philanthropy expanding the number of cause areas that it works in two- or threefold over coming decades, or is it likely to be that to add one you have to take one out?

Holden Karnofsky: Oh, I think there’s a really good chance that we’re going to increase the number of focus areas we work in. Just as we increase our giving, we could do it by increasing the budgets of current areas or going into new ones, and I imagine it’ll be a combination.

Robert Wiblin: What are kind of the key themes that tie together all of the focus areas that you’ve chosen and the focus areas that you almost chose? Why is it that these causes stand out in particular?

Holden Karnofsky: We went out there to choose causes based on importance, neglect, and tractability, so that’s the kind of direct answer, and we didn’t really optimize for anything other than that. But I would say that having chosen a bunch of these causes, I think I have noticed a couple of broad buckets that most of them seem to fall into. There are kind of two theories of how philanthropists might really be onto something big today that isn’t getting enough attention. They can have a really outsized impact. One theme, and we have a blog post about this idea, is this idea of what we call “radical empathy”, which is this idea that a lot of the worst kind of behavior in the past, when you look backward and you feel really bad about certain things like just attitudes toward women and minorities and slavery and things like that, a lot of it could be characterized as just having too small a circle of concern and just saying, “We’re going to be nice to people who are sort of in our tribe, in our club, in our circle. But then there’s this whole other set of people who are not counting as people, and we don’t think they have rights, and we think they’re just different, and we don’t care about them, and we treat them as like objects or means to an end.”

I think if you could go back in time and do the best philanthropy, a lot of it would be trying to always be working with a broader circle, always be trying to help the people who weren’t considered people, but would later be considered people. So for example, working on abolitionism or early feminism or things like that. And a lot of our philanthropy does seem to fall into this category of we’re trying to help some population that many of the wealthy people today who have the power to be helpful just don’t care about or don’t consider people in some sense, or just aren’t really weighing very heavily, are highly marginalized.

And so criminal justice reform is certainly an example of this. I mean, people affected by our over-incarceration disproportionately are sort of low income or disproportionately minority, and in general, I think also people think about offenders, or people think about incarcerated persons and think, “I don’t care about them. I don’t care about their rights. Why should I care about their suffering?” And so that is kind of a cause where we’re trying to help a population that I think is quite marginalized, and that makes the dollar go further.

You could say the same thing about global poverty. The GiveWell top charities is kind of a similar situation where a lot of people believe charity begins at home. Of course, some countries are much richer than others, so the home of the rich people gets a lot of money, gets a lot of charity, and the home of the not-so-rich people doesn’t. You just have a lot of people in America who believe that Americans always come first, and they don’t care about people in Africa, and so you get this opportunity to do extra good by helping an extra-marginalized population.

And then some of the other work we do is going a little bit further in that direction, to the point where it gets legitimately debatable, in my opinion. The farm animal welfare work, I think a lot of people … I mean, I think the reason we’re seeing such amazing opportunities in farm animal welfare and such sort of high leverage places to get these really quick wins that affect a lot of animals is because most people just don’t care about farm animals. They say, “Chicken? Well, that’s dinner, that’s not someone I care about. That’s not someone with rights.” We believe that it’s possible we’ll all look in 100 years and say, “That was one of the best things you could be doing, is helping these creatures that we now realize we should care about, but at the time we didn’t.”

That’s definitely a common theme that runs through a lot of what we do, and then the other theme is … it’s very related to long-term-ism, and it’s this idea of kind of X factors for the long-term future. This idea that there could be dramatic societal transformations that kind of affect everyone all the way into the future. An example of this is like if you had gone back 300 years and you were trying to do charity or philanthropy just before the Industrial Revolution, I think the Industrial Revolution ended up being an incredibly important thing that happened that had incredibly large impacts on standard of living, on poverty, on everything else. I think in retrospect it probably would have been a better idea to be thinking about, if you were in the middle of that or just before it, how it was going to play out and how to make it go well or poorly, instead of, for example, alms. Instead of, for example, giving money to the low income people to reduce suffering in the immediate situation.

And so when we look at global catastrophic risks, and to some extent a breakthrough science as well, we’re looking at ways that we can just affect an enormous number of persons by kind of taking these things that could be these high leverage moments that could affect the whole future. And so our interest in AI, our interest in pandemic, our interest in science, a lot of it pertains to that.

Robert Wiblin: I know you’ve been working on a series of articles about the history of philanthropy, looking at really big wins that philanthropists have had in the past, and I guess potentially some failure stories as well. Is there anything you’ve learned about that in terms of which focus areas tend to be successful and which ones tend not to?

Holden Karnofsky: Yeah, I think we’ve learned a ton from the history of philanthropy, and we have a whole bunch of blog posts about it, and we’ve started a little bit of a grant-making program where we actually fund or contract with historians. What we initially did is we just took this casebook. This “Casebook for The Foundation: A Great American Secret” is, I believe, what the book is called, and it’s got these two-page sort of vignettes of a hundred different philanthropic successes. I kind of read through them and looked for patterns and made a blog post about that, and then what we started doing is taking some of the more interesting-seeming ones and asking, and paying historians to go and really check them out and say, “What really went down here? Was philanthropy really that effective and really that important?”

And then there are some others we’ve been able to learn about independently, just by reading books and whatnot. Yeah, we’ve learned a ton. One of the philosophies that I think history of philanthropy has pushed us toward is what we call “hits-based giving”, and that’s the idea … again, it’s a very different school of thought from GiveWell … the idea of hits-based giving is a little bit similar to venture capital, which is that rather than trying to exclusively fund things that you think will probably work, what you do is you fund a whole bunch of things that individually, each one, might have, let’s say, a 90% chance of failing and a 10% chance of being a huge hit. And then out of 10, you might get nine miserable failures, one huge hit, and then the huge hit is so big that it justifies your whole portfolio.

I was definitely pushed toward this vision of giving by reading these case studies, because even though the case studies are cherry-picked, when you see some of the examples I saw, like the Green Revolution, I think one of the most important humanitarian developments in all of history, certainly in the last century, and looks like it was largely a philanthropy story. When you look at the pill, also just incredibly revolutionary and important, and largely the result of a feminist philanthropist. You know, you just say, “Boy, I could fail a lot of times. If I got one of those hits, I would still be feeling pretty good about my philanthropy.”

I think when I read through this stuff, I was initially surprised at how successful things had been, just how big some of the big successes were, and that’s not to say there’s no failures, because I think there’s a lot of failures. So that was one of the first things we took away, also a lot of the interest in scientific research, a lot of the interest in policy and advocacy. Those are things I think philanthropy really had a track record of doing well in. More recently Luke Muehlhauser did a study of how philanthropy has contributed to the growth of academic fields. I think there’s actually … And one of the things I learnt from it, is many times throughout history, let’s say there’s a topic that just for whatever reason it’s not really being discussed much in academia. It’s not something professors work on. It’s not something students work on. Professors unlikely to switch into it, because they kinda already have their specialty. Students are unlikely to switch into it, because there’s no professors in there. And then philanthropy comes along, one example is the field of geriatrics.

But philanthropy comes along and says, “We think this should be a major academic field.” And through fund kind of all parts of the pipeline, professors, students, conferences, you name it, we believe they’ve really been a big part of causing these fields to grow. And so we’ve tried to look at what they did and learn from it. ‘Cause we look at the situation today and we say, “Boy, we wish there was more of a field,” for example, “Around the AI alignment problem.” There’s a lot of academic researchers working on machine learning and making AI more effective. But in terms of making AI more reliable, more robust, safe, in-line with human intentions, we wish there was more of a field there. So I think it’s been a major source of learning for us. I’ve really enjoyed the history of philanthropy case studies that the historians we’ve worked with have written, and just they give me an enriched picture of what has come before and what we can learn from.

Robert Wiblin: Do you just wanna flesh out those stories of the green revolution and the invention of the pill? ‘Cause I had no idea that they were funded by a philanthropist. And you might expect that I might know. So-

Holden Karnofsky: Yeah. Yeah.

Robert Wiblin: … perhaps [crosstalk 00:32:32] have no idea.

Holden Karnofsky: Yeah, sure. On the green revolution, I believe this began with the Rockefeller Foundation essentially funding a team of agricultural scientists, among them Norman Borlaug who would later win The Nobel Peace Prize. And the idea is that they were looking to develop better crops for the developing world. So kind of U.S. agriculture or whatever developed world agriculture was good at kind of breeding and optimizing crops for the climates they were in. And they wanted to provide kind of a similar service for poorer countries. And what happened was, they developed these crops that were incredibly productive and incredibly sort of just economically fruitful to work with. And it originally was in Mexico.

But one of the things that I believe happened as a result of this eventually scaled up, and they started optimizing them for a whole bunch of different countries and climates. And India at the time this happened was kind of in the midst of a famine. And I believe they went from being a wheat importer, to being a wheat exporter. And then there was just this massive growth that was kicked off beginning with the agricultural sector, that then leading to nationwide growth in several countries that are now held up as some of the big examples of countries coming out of poverty in the 20th century. And so it’s hard … I mean, there have been estimates that a billion deaths from starvation were prevented by these kind of new and improved crops, and all the developments that came from that.

And it’s really hard to think of something that’s been more important for human flourishing. There are things you could say, but this really started with a foundation. And I think it’s interesting, because at the time I don’t think that was the kinda thing that a government funder would necessarily do. So this comes back to my example of how philanthropy can play a really special role in the world. I think governments often, they have a lot of resources but they maybe kind of bureaucratic, and they may be kind of fighting yesterday’s battles in some sense. Doing the things that are already socially accepted. And I think global poverty reduction was not as big a worldwide priority then. And so philanthropy kinda led the way.

The pill too. Katherine McCormick was a feminist philanthropist. And the feminist Margaret Sanger came to her with this idea that she knew a scientist who had been working on studies with rabbits to see if he could get them to basically control the menstrual cycle, and control fertility. And Sanger believe this could be a huge breakthrough for feminism just to give women more options, and let them live their lives the way they wanted without having all the biological constraints they had at the time. And again, not the kinda thing government was really excited about. In fact I believe that they … Initially they couldn’t advertise this as a birth control. I think they advertised it as something else, and then they put a warning label on it saying, “May prevent fertility.” I think they’re actually [crosstalk 00:35:07]-

Robert Wiblin: … prevents period pain. Or something like that.

Holden Karnofsky: Yeah. Exactly. Yeah. And I think they were required to put the label on there. But that label was effectively their advertising, the warning label. So it’s another example of something that it wasn’t really a major society-wide priority. It wasn’t really a government priority. It was at the time of something new and edgy, and different. And I think again, it was like you look at what happened. And there was this neglected research on rabbits and fertility. And it got funded by just a private philanthropist, not a huge amount of money. And boy, did it change the world.

And so when you look forward, you gotta ask I mean, what are the things that just, they’re not a world-side priority today, but if the world becomes sorta better off and wiser, and more cosmopolitan, maybe they could be in the future. So the way that animals are being treated on factory farms. This is not a giant hugely popular cause. There’s not a lot of government funding trying to do anything about this. And so this is another case where maybe starting with private philanthropy, we start to turn the tide. And so yeah, those are kind of some examples.

There’s philanthropic examples that are much more, I would say debatable in their impact. So there’s a book called, The Rise of The Conservative Legal Movement, but Steven Teles. Where it’s argued that conservatives did a really outstanding job of philanthropy in terms of effectiveness. And they really changed the public dialogue in the U.S. kind of for the good, or for many decade. Which they explain in some ways some of the strange occurrence of intellectual activity in the U.S. And some people think this is amazing, and some people think it’s terrible. But it certainly was a big deal. And so I think if you’re looking for a lot impact for your money, trying to think about how some of these hits came about is pretty fruitful.

Robert Wiblin: Have you learned any other lessons that you wanna point out from the history of philanthropy?

Holden Karnofsky: Yeah. I mean from the field building exercise, when we talk about failures I mean, we focus more on successes, ’cause we’re hits based. And we’re trying to understand, there’s gonna be a lot more failures than successes. And there’s a lot more ways to fail than there are ways to succeed. So we tried to understand the successes. But the field building was pretty interesting too, because there are a couple examples of fields where it looks like someone tried to build a field in a way that actually stunted the growth of the field.

So nano-technology, and I believe cryonics might’ve been held up as examples of this in Luke’s report where, by coming into the field and creating a lot of hype and media coverage without really building strong connections to the scientific community, people kind of made certain topics taboo and illegitimate. And took things that could’ve had a decent scientific foundation, and made them just like impossible to work on in academia. That’s a great example of the mistake we really don’t wanna make with our research, safety, alignment. So this was something I was really glad to know about.

Robert Wiblin: Yeah. I did another episode with Professor Inglesby who mentioned that if you come into an area with an interest in one particular area of a field and saying like, “This is much more important than everything else in the field,” you can potentially really alienate everyone else. Because you’re basically saying, “I’m gonna try and cannibalize all of your people and all of your grant funding.” And so even if you do think that the subset of a field that you’re particularly interested in is more important, maybe you shouldn’t announce that, and shouldn’t suggest any hostility to others, because you turn people who otherwise might be supporters or at least neutral into adversaries. Is that something that resonates with you?

Holden Karnofsky: Yeah it certainly sounds possible. I mean it’s very hard to make all these generalizations I think. But I can certainly see it happening. And I think one of the things that makes it worse is if you’re letting the media and the hype get ahead of the science. I think that does seem to be, intuitively and from these couple of example, like a good way to antagonize scientists. And ultimately the scientists, they’re the ones who determine who’s getting tenure there. They’re the ones who determine what’s a good career. In my opinion, pinning the media against a scientist is not really a desirable situation.

Robert Wiblin: Okay. So that’s something about choosing causes, and I guess ways of tackling those problems. Let’s talk about hiring staff. How do you go about hiring?

Holden Karnofsky: Well, one of the things that we feel is that interviews are incredibly unreliable. I think there’s research on this. It’s also been our experience. What we wanna do is, we wanna try and simulate working with someone to the greatest extent possible. And so a lot of times when we’re hiring, we try and do these little work trials where someone will do an assignment that has some value to us. It’s often not the most important thing on our plate, ’cause we don’t wanna be reliant on the work trial to get it done. But something that has some values to us, and we can kind of simulate working together in a collaborative realistic style. And I think we’ve written some, so we have a couple blog posts about how we hired our first program officers. And I think that’s one of the most important decision, like I mentioned, that a foundation makes. Especially the way we operate.

I mean I mentioned that our grant making process … Our grant making process is really dependent on having one person who’s very deep in the field, big time expert, and knows what they’re doing, and is creative and can propose great things. And so hiring the wrong person would be a big mistake there. And so we did a pretty intensive process to hire our program officers, and a lot of what we looked for is, we looked for people who could kind of communicate in a very systematic way. So that we understood what arguments they were making and why they wanted to fund what they would fun.

We also looked for people who are incredibly well connected and well-respected in the field. Because we knew that their greatest source of input was gonna be from other people in the field, other experts. And a lot of the key was gonna be to get those people to open up to them, despite the fact that a lot of those would want funding. And finally we looked for people who are very broad. So when we interviewed people for the program officer role, we spoke to some people who kind of, they had been doing one thing and they knew how to do it. Whether that’s grassroots advocacy, or grass-tops advocacy for example. They knew their thing, but they weren’t kind of thinking about all the different pieces of their field, and all the different things you could do to push their cause forward, and how they fit together. So we looked for breadth as well.

Robert Wiblin: Yeah. How much autonomy do program officers have? Do you usually approve the kinda grants that they suggest? Or is it a bit more competition for influence?

Holden Karnofsky: Sure. So the Open Philanthropy grant making philosophy again, inline with hits based giving. Inline with trying to take big risks, and do things that could be amazingly good even if they might fail a lot of time. One of the things that we wanna do is minimize the number of veto points, minimize the number of decision makers. And have kind of the person who knows the field best has the most expertise, really be in the lead of what they’re doing. And that said, we don’t wanna be too absolutist about it, because we don’t have perfect confidence in our hiring decisions and also philanthropy as like I said, it’s not necessarily a very intellectually developed field. So most of the people we’ve hired did not previously work in philanthropy. They have a lot to learn.

And so we also need to kind of hold ourselves accountable and understand where the money’s going. And so one of the compromises we found is what we call the 50/40/10 rule, which is that … So we take a program officer and we look at their portfolio, which is all the grants they’ve made. And one of the things we ask them for is that we’d like 50% of the grants by dollars to be classified as good, and by, “Good,” I mean that the decision makers, the grant approvers, which are myself and Cari Tuna, kind of affirmatively have been convinced of the case for this grant. That if you sort of took for example out grant to Alliance for Safety and Justice, and you start arguing me about it, I feel like it reasonably defend the grant. And you would sort of own it in a sense.

But then another 40% of the portfolio, so bringing us to 90% total is okay if we mark it as only okay. And what, “Okay,” means is like, we can see where the person’s coming from. We can see how if we knew more, we might be convinced it’s a good grant. We don’t necessarily buy all the way into it. And then the other 10% of the portfolio is what we call discretionary where there’s actually a very stripped down process. And a program office can just send a very short email. And unless we see a major red flag or a downside risk, it goes through in 24 hours if we don’t respond. And we don’t have to have any buy in to the grant.

And so the idea there is that it’s trying to strike a balance. So we look at our portfolio, and half of it we kind of understand, and it’s been argued to us and it’s been justified using our somewhat thorough internal writeups. And then the other half of it is more like, it’s an opportunity for the program officer to take the things that would’ve been harder to convince us on, because they took more context and more expertise. And just not put in this much sweat, and just get them done anyway. And so that’s where we’ve really tried to empower the program officers in that way.

Robert Wiblin: So in the context of still giving advice to a hypothetical billionaire who’s starting a foundation. What are the biggest likely mistakes that they might make in the hiring? How would they end up hiring someone who was a real mistake?

Holden Karnofsky: I think two of the biggest mistakes during made in hiring, I think one thing is that … I mean, these both relate to the things I’m saying about how we look for a program officers. But I think that … I mean, you’re probably not hiring someone who’s doing philanthropy before. And if you do, you’re probably hiring someone who’s doing it under much more constrained environment, under a foundation that kind of had some pre-declared are of focus and knew exactly what it wanted. So, the person you’re hiring is probably not exactly experienced in exactly the thing that you want them to do.

And I think one danger is that they’re just gonna keep doing what they used to do. Keep doing what they’re used to doing. And that’s why we think it’s really important to look for breadth. And I still worry about this even with the people we have, ’cause I think it’s just, it’s very easy. Let’s say that you’re working on farm animal welfare, and you work vegan promotion. Convincing people to go vegan. Or you work kinda corporate campaigns, which is convincing corporations to treat animals differently. Those are very different activities. And a lot of the times with people working on the two things, they don’t always have a ton of interaction, they don’t always have a lot of common intellectual ground.

And you have to ask yourself, is your program officer, are they funding only one of them because that’s what they know and that’s what they like? Or they’re funding only one of them, because they made a calculated decision? Or they’re funding both of them, because they’re trying to just compromise with everyone and be in everyone’s good side? Or they’re funding both of them for good reasons? So I think having people who can really look at all the different aspects of a field I think is very important.

And then I think another major … Another thing that I kinda just like actually worry the most about with our program officers is, I think it is a really challenging dynamic in philanthropy. And I think this makes it a bit different from other kind of areas that might be considered reference classes for it is that, you’re out there trying to make intellectual decisions and decide what to do. But practically everyone you’re getting advice from and feedback from, and thoughts from is someone who either now or maybe in the future is hoping they’ll get money from you. And so they don’t wanna be on your bad side.

And so there’s this kind of saying, this kind of joke that once you become a philanthropist, you never again tell a bad joke. Because everyone’s gonna laugh at your jokes whether they’re funny or not. Because everyone wants to be on your good side. And I think that can be a very toxic environment. I mean I personally am a person who really prizes openness, honesty, direct feedback. I really value it. I really value people who criticize me. But a lot of people that I interact with don’t initially know that about me, or maybe just never believe it about me.

And so if someone is worried to criticize me, I may unintentionally just be doing the wrong thing, and never learn about it. And so I think one of the worst qualities a program officer can have is being someone who people won’t tell the truth to. Being someone who doesn’t take criticism well. Being someone who’s overly pushy, overly aggressive. And kind of is always using the fact that they have some kinda power and they’re able to give money, to always kinda be the person who’s getting praised and getting complimented. And laying things out the way they want to be. And be very bad at sort of doing this active listening and encouraging people to share thoughts with them.

And I think it’s a huge challenge. I mean I think it’s somewhat related to the management challenge of, how do you get people to show their honest thoughts with you when they might fear criticism. And we at least want program officers who worry about it a lot, and who form close enough relationships that they’re able to hear the truth from many people. And if they don’t, I mean I wouldn’t be very optimistic about the philanthropy there.

Robert Wiblin: Given that the program officers, they’re trying to get quite a lot of money out the door. Potentially tens of millions each year. And they might wanna make grants to a dozen, maybe more organizations. And they don’t wanna say, “Yes,” to everyone. ‘Cause then they’re not adding that much value. So they must look into potentially dozens of organizations each year. And they have to follow up on the previous grants. I mean how do they find the time to manage all of this stuff and actually still be like somewhat thorough?

Holden Karnofsky: Sure. How do program officers find the time. I mean, they do work really hard. It is a bit of a … It’s a job with huge opportunities to do a lot of impact. And it’s also very challenging job. We’re also I think in the process of just staffing up a bit to take some of the load off of them for grant renewals, grant check-ins. So some of our program areas already have sort of more than one person. Where we’ll have an associate reporting to the program officer and helping them out. And some of them I think we’re gonna get there in the future.

But you know, the other thing I’d say is that I think a lot of the ideal way to be a program officer is to be very network dependent. And so I think one of the things that we encourage people internally to do, is to spend a ton of time networking, talking to people. We definitely encourage them to go to dinner with people, and expense it and all that stuff. Because I think, let’s say you work on farm animal welfare. If you know everyone else who works on that topic, and you kind of … All the people are doing the best work. I mean you don’t have time to be friends with everyone. But you know all the people who are doing the best work and have the most thoughts. And you have good relationships with them. I think you can learn a lot just by talking and brainstorming.

And then a lot of the … It’s gonna happen if you do that well, in my opinion is you’re gonna hear a lot of the scuttlebutt about what the different organizations are good at, what they’re not good at. Who’s the best at each thing, what the biggest needs in the field are. And so that can form a lot of the basis for how you ultimately decide what to prioritize, what to choose, whom to fund. Of course, one of the things that makes this challenging for Open Philanthropy and very different from GiveWell is that, I think a lot of the most valuable information here is exactly the information that people will not tell you, unless they trust you.

It’s information about just who’s good and who’s not so good at they do, and what the biggest needs in the field are. And sometimes also information about how to deal with adversaries. So when you’re doing political advocacy, you often have people who are against you. And so the information people will only tell the people they trust, is the key information behind assessing a grant. And a lot of times it comes from a lot of just interpersonal, getting to know people, building trust, forming opinions of people that is not only delicate information. It’s information that even if you wanted to explain it all, you couldn’t.

And so that’s why Open Philanthropy really had this emphasis on, we don’t only wanna fund that which we can explain in writing. That’s why we have the 50/40/10 rule. Like we don’t wanna only fund that which Holden buys into. And that’s also why we don’t have the same approach to kind of public communications that GiveWell does. We don’t try to explain all the reasoning behind each grant. And it does … It’s pros and cons. I mean I think this frees us up to do really creative high-risk great things. And I think it also, it makes our work in some ways less satisfying, less thorough, less easy to kind of take apart into its component pieces than GiveWell.

Robert Wiblin: Is there a tension between wanting to be friends with these people and get their honest opinions, and perhaps a bit of gossip out of them as well. And potentially also having to kinda crack the whip with the people who are giving grants to about their performance? I mean, how much do follow up. And yeah, pay close attention to potential mistakes that they’re making?

Holden Karnofsky: Yeah. I think it’s a huge tension. And I think this is one of these fundamental challenges of philanthropy that I wish I had more to say about what to do about it. It continues to be an ongoing topic for us. And there is certain things that we’re experimenting with today to try and do a better job. Just evaluating ourselves, and seeing are we getting the best information from everyone? And also do we have good relationships with everyone? I think one place that I kind of analogies it to is management.

So I think management has some of the same challenges. You’re trying to help someone, you wanna know how they’re doing, you want the truth from them. You’re also responsible for evaluating them. That has some relationships to a philanthropist-grantee relationship, but there’s also some important differences. So it’s just an example of something that I think is kind of this open question in philanthropy. I can’t point you to a lot of great things to read about it. But I think it’s a really important topic. And it’s one of the things that we want to develop a better view on and share that view with other philanthropists.

Robert Wiblin: So you said that one of your goals is to make philanthropy as a whole a bit more of an intellectual exercise. And I guess, you’ve been writing up some pretty long blog posts about your process for deciding what problems to work on, and how you decide how to split resources between different problems. I’ll stick up links to those. Is that the main reason why you’re writing those things up in such great detail as to see if you can change the discussion among other foundations?

Holden Karnofsky: Well, one of the reasons why we’re writing them up in such great detail is because it just, it helps us get better feedback. And it helps us get clear about our own thoughts. So when a decision’s important enough, I believe that it is good to just write it down in a way that you’re not using a bunch of esoteric inside language that you might be bearing implicit assumptions in there that makes sense to you and the people you talk to, but not to others. I think just the process of trying to make it clear and trying to make it for a more general audience somewhat raises new questions and clarifies the thinking.

And then I also think we’ve gotten good feedback on some of these thorny questions we work by sending them to people, especially in the effective altruistic community for comment. But yes, another thing we’re trying to do is document a lot of the thinking behind the tough intellectual decisions we’re making. And I think at some future date, I’m hoping that that content, we may have to clean it up. We may have to present it differently, ’cause currently it’s like it’s almost academic in tone. It’s just very dry. So I don’t think we’re in a place right now where this stuff is getting into the press and all kinds of philanthropists are reading it, and changing what they do.

But we do send it to people we have individual contact with. And I could imagine that in the future, it’ll become this sort of, the intellectual backbone of some more presentable presentation of what we end up settling on is the right intellectual framework for philanthropy. And we’re definitely not there yet.

Robert Wiblin: Let’s talk about one of those more difficult decisions that you have to make as a-

Holden Karnofsky: Yeah.

Robert Wiblin: … foundation. Which is how you split the total kind of endowment between different focus areas. What is the process that you use for doing that?

Holden Karnofsky: Sure. So this is something we’ve struggled with a huge amount. So one of the questions we struggle with that has to do with this is when we’re looking at a grant, and we’re trying to say, “Should we make this grant or not?” One way to frame the question … I mean once you’ve put in the time to investigate a grant. One way to frame the question of whether you should make the grant is, would the money do more good under this grant, or under the last dollar that we will otherwise spend? And so in other words, if you imagine that you’re gonna make a $100,000 grant, you can either make the grant or you can not. And if you don’t, then you have a 100,000 extra dollars. And in some sense that’s gonna come out of your pot of money at the end.

And so we call this the last dollar question. And we try to think about each grant compared to the last dollar we’re gonna spend. And this becomes a very challenging problem, because initially the way we wanted to think about it, is we wanted to just kind of say, “Let’s estimate. Let’s sort of have a model of how much good per dollar the last dollar does. And let’s take each grant and have a model of how much good per dollar that grant does. And then when a grant looks more cost-effective than the last dollar, we’ll do it.”

And that brings us into this domain of making these kind of quantitative estimates of how much good you’re doing with a grant. And I think there’s a lot of problems with trying to make these quantitative estimates. And some of them are very well-known. But I think some that have been I think a little but unexpectedly thorny for us is what we call, the worldview split problem.

And so the way to think about that is, let’s say I’m deciding between a grant to distribute bed nets and prevent Malaria. And a grant to fund cage-free campaigns that will help chickens. And let’s say that I find that I can either sort of help, let’s normalize and let’s just pretend that help is always the same amount. We could help a person for a $1,000, or we can help a chicken for $1. Help a 1,000 chickens for a $1,000. Which of those is more cost-effective? And I think it really comes down to this incredibly mind-bending question, which is, do you think chickens can feel pain? Do you think they can have good lives? Do you think they count as sentient beings? Do you think they have rights?

How do you think about the experiences of chickens and how much you value the lives of chickens compared to persons? And I think maybe the initial answer here is, “Well, I don’t care about chickens, I care about people.” But I think the more you think about it, the more this becomes a kind of a mind-bending question. Because if you care about people more than chickens, on what basis is that? Is it that you value sophisticated behaviors? Is it that you value people in your community? And why is that? And which of these things might apply to chickens?

And the problem is that if you end up deciding that, let’s say you value chickens 10% as much as humans, that would tell you that the chicken grant is much better. Let’s say that you don’t value chickens at all. That would tell you the human grant is much better. And so when you’re trying to make these estimates of how much good you’re gonna do, or how many people you’re gonna help, or how many persons you’re gonna help per dollar, a lot of it is just there’s this small number of really tough mind-bending philosophical questions that if you go one way, it says you should put all your money into farm animal welfare. If you go another, it says you should put all your money into global poverty. And maybe if you go a third way, it says … I know you’ve had another podcast on this, that you should put all your money into sort of [long-termism 00:55:07]. Because there may be a lot of chickens in the world, but there’s even more sort persons in the future than there are of any kind in the world.

And so that is a really tough one for us and for a variety of reasons, which we’ve written up in some detail on the web. We haven’t been comfortable just picking our sort of mid-probabilities and rolling with them, because we haven’t been comfortable with the idea that all of the money would just go into one sort of worldview, or one sort of school of thought of giving. And one of the reasons we feel that way is, we do believe there’s a big gap in the philanthropy world. And we believe that we have special opportunities to help on a whole bunch of different fronts. So we think we have special opportunities to help reduce global catastrophic risks. We think we have special opportunities to help animals. And we think we have special opportunities to help people.

And so to leave really outstanding opportunities to kind of help and change the way people think about things on the table, in order so you can put all your money into just one kind of giving, has not been something that’s sit right with us. And that’s something that we’ve written about. So we’re still … This is just a … This question is just very much in-progress. And we have this blog post kind of laying out how we’re now trying this approach where we kind of, we think about our giving in a whole bunch of different levels at once.

So we think about each grant and how good it is according to its own standards. So we think about how much a farm animal welfare grant helps chickens, and how much a global poverty grant helps human. And how much a global catastrophic risk reduction grant reduces global catastrophic risks. And then we also have to have sort of, an almost separate model of how much money we wanna be putting behind the worldview that says global catastrophic risks are most important. How much total we wanna be putting behind the worldview that says that humans are most important. And how much behind other worldviews.

And so I think we have a long way to go on this. And I think we have a lot of work to do both philosophical and empirical. And one of the reasons that we’re on a hiring push right now is I think, this could be a decade’s long endeavor to really … And there is no right answer. So the idea is not that there’s gonna be like one set of numbers that is in some sense correct. But just to come to a very well considered positions on this stuff where we feel we’ve considered all the pros and cons, and we’re making the best decisions with this very large amount of money that we can possibly make under deep reflection and under being highly informed. That’s just a ton of work. And I think it’s gonna be a very long project for us. And we’ve just kinda gotten started on it.

Robert Wiblin: So I’ll put up a link to your latest blog post from last month about this topic of how, yeah, how you try to split the money between the different focus areas. One thing that occurred to me when reading it is that you kinda layout three archetype or focus areas, one is human welfare today, one is animal welfare today. And another is the long term future, or humans around in the future. Do you worry that the split could end up being influenced by kind of an arbitrary way in which you’re categorizing these things?

So you could divide them up differently. You could have humans alive today, animals lives today, and then humans lives in the 22nd century. Humans alive in the 23rd century. Humans alive in the 24th century. Or you could split up humans alive today by country or something like that. And it’s entirely clear that like there’s three clusters here, or that’s the natural way of cutting it up. And perhaps like by defining three different things, you are suggesting that well, that [inaudible 00:58:14] start adjusting from is one third for each of them. Do you see my concern?

Holden Karnofsky: Oh, sure. Yeah. No, I definitely think so. I mean I think that one of the challenges of this work is that it’s, we’re trying to kind of list different worldviews, and then we’re trying to come up with some sort of fair way to distribute money between them. But we’re aware that the concept of worldview is a very fuzzy idea. And it could be defined in a zillion different ways. And I mean I can tell you how we tried to tackle it so far. I mean first off, there’s questions we struggle with where it feels like there’s just like part of me wants one answer, and part of me wants another answer. And it’s just an intuitive thing. That it’s just, I don’t feel like I struggle a lot with how I value the 25th versus the 26th century. I think I probably should either value both of them a lot, or just like …

Or for one reason or another, many of them epistemological rather than philosophical, for one reason or another discount those very far centuries that I have a very hard time understanding or predicting. And same with animals where I feel like there’s kinda one side of my brain that wants to use a certain methodology for deciding how much more a weight to give to chickens, and how much more a weight to give to cows and all that stuff. And it’ll use that methodology.

And then there’s another side of my brain that says, “That methodology is silly,” and we shouldn’t do radically unconventional things based on that kind of very weird suspect methodology of pulling numbers out of thin air, on how we value chickens versus humans. So some of this is just purely intuitive. And a lot of philanthropy ultimately comes down to that. I mean in the end as a philanthropist, you’re trying to make the world better. There’s no objective definition of better. All we can do is we can become as well-informed and as reflective, and as introspective as we can be. But there’s still gonna just be these intuitions about what the fundamental judgment calls are, and what we value. And that’s how we’re gonna cluster things.

Another way in which we do decide how to set up these worldviews is just practical. So some of why I wouldn’t be very happy with all of the money going to animal welfare, is that I think that is a … Animal welfare is kind of a relatively small set of causes today. And there are a lot of idiosyncrasies that all those causes have in common. And so if we became all-in on animal welfare, our giving would become very idiosyncratic in certain ways. And I think in some ways you could think of that as like muddying the experiment of doing effective philanthropy and trying to help effective philanthropy catch on.

And so the bottom line is like, I don’t have a clean philosophical way of seeing exactly what all the questions are, and exactly how much money should go everywhere. But what I can do is, I can say, there’s a point of view that says that we should optimize our giving for just the long-term future and for reducing global catastrophic risks. And I notice that if we put all the money into that view, we would have certain problems both practical and just intuitive, philosophical perhaps, problems. And so maybe we want some sort of split there, and then I can look at a similar split with the animal stuff.

So it’s a mix of practical things and just very good intuitive things. But yes, we don’t believe that we’ve got any objective answers to how to define, “Good,” how much good is being done per dollar. But we do believe what we can do is be very informed, very reflective about these things. And consider all the arguments that are out there. Consider them systematically. Put our reasoning out so that others can critique it. And we think that’s a big step forward relative to the default way of doing philanthropy, which is really just to go with what kind of … Before any reflection, before any investigation feels interesting, and passion compatible to you. So we think it’s a big improvement. But yeah, we’re not … We can’t turn this into a real science. There’s no way.

Robert Wiblin: When it comes to deciding the splits, do you find in your mind that you start with a kind of an even split, and then move from there? Or do you start from, “We should give a 100% of it to the most cost-effective one.” And then kind of adjust down from there based on considerations that suggest that you should split more evenly?

Holden Karnofsky: Well, sort of the question here is what does cost-effectiveness means. So I mean certainly in general, if we could agree on a definition of cost-effectiveness, I would always wanna give to the most cost-effective thing. The question is more about methodology. The question is more about, do you feel that the right way to estimate cost-effectiveness is to, for example, write down your best guess at the moral weight of each animal, and then run the numbers, or do you feel that it’s to allow certain more kind of wholistic intuitions about what to do enter into your thinking? So yeah, I mean, I think the question is about what cost effectiveness is. I’m definitely inclined to do the most cost effective thing if I can figure out what feels cost effective to me. And cost effective just means good accomplished per dollar, and so it’s entangled with the idea of what’s good, and that’s the intuitive idea.

That said, I mean, I think, I guess the intuition does sort of start in a 50/50 place although we’ve tried to list the reasons that it shouldn’t be 50/50. So I think one of the concepts we talk about in our post is let’s say you’re deciding between the sort of long termist view that you should optimize the long term future, and the near termist view that you should optimize for kind of impact you’ll be able to see in your lifetime. You can ask yourself one question, which is, if I reflected on this an inordinate amount, and if I were possibly way more self-aware and intelligent than I actually am, what do I think I would conclude, and what’s the probability that I would end up deciding that long termism is correct or near termism is correct? So that might give you not a 50/50 split. That might give you like an 80/20 or a 70/30.

And then there’s these other things we’ve put in there too, which is like, you should take into account, you know, perhaps even if one worldview is less likely, there’s more value to be had if it’s correct. There’s more outstanding giving opportunities. There’s a lot we’re putting in there, and certainly, we’re not just gonna end up going even split, but I think there is some intuition to go there by default, though we try to counteract that listing counter considerations.

Robert Wiblin: If you think that there’s really sharp declining returns in some of these kind of boutique focus areas, like perhaps artificial intelligence or factory farming, then that could suggest that in fact it doesn’t really matter exactly what kind of split you get between them. As long as you’re giving some to those focus areas, then that’s where most of the [inaudible 01:03:46] is gonna get done. So is it possible that maybe you shouldn’t spend too much time fussing about the details, if you’re gonna kind of split it somewhat evenly anyway?

Holden Karnofsky: I think in some ways that was the attitude we took at the very beginning when we were picking causes. And I think that attitude probably made more sense when we were early, and we just, we wanted to get experience and we wanted to do things. We didn’t wanna hold everything up for the perfect intellectual framework. And we’re certainly not holding things up. We’re giving substantial amounts. But I think at this time, I mean, I think it is true that we can fund the best opportunities we see today in AI and in biosecurity, without getting that close to tapping out the whole amount of capital that’s available. That is certainly true, but there’s a question a, of what kind of opportunities will exist tomorrow. So one of the things that we do as a philanthropist is we do field building, and we try to fund in a way that there’ll be more things to fund in the future than there are in the present. So that’s one complication there is just imagining. There may actually be a lot of things to spend on at some future date.

And then another thing is, you know, one of the things that I find kind of mind bending is thinking about there’s probably something you can do for every worldview. They may not be as good as the best opportunities, but it’s still something. And so if you believe, as you covered in other podcasts, if you believe that there’s so much value in the long term future, you might believe that relatively sort of intuitively low impact interventions, like let’s say directly funding surveillance labs in the developing world to prevent pandemics. You might still believe that in some mathematical sense is doing more good in expectation and helping more persons than even very effective sort of science that’s wiping out very terrible diseases. And so what you do with that observation, when you could do something that really feels intuitively outstanding and impressive, and helps a lot of people, versus something that feel intuitively kind of low leverage, but according to some philosophical assumptions could actually be much better. And that’s one of the things that we’re struggling with at this time.

Robert Wiblin: So what kind of wins have you had so far? Are there any things that you’re particularly proud of?

Holden Karnofsky: Yeah, for sure. It’s early days, so it’s only been a couple years that we’ve really been giving at scale, and I think, in general, I think of the timeframe as, for philanthropy, as being somewhere between like five and twenty years to really hope to see impact. So we’re not at a stage where I would necessarily be, let’s say demanding to see some of our hits yet. But I do think we’re starting to see the early hints of things working out, and kind of this [inaudible 01:06:01] framework. I think the area where that’s been most true is farm animal welfare. That’s been a really interesting case because I think it’s an incredibly neglected cause, and so, you probably talked to Louis about this, but there are these animals being treated incredibly terribly, and they could be treated significantly better for very low costs on the part of the fast food companies and the grocers. So going from caged chickens, battery cages, to cage free chickens is a very low cost thing. It’s like a few cents a dozen eggs or something like that. And just the corporations don’t do it because just no one cares, and no one is even bringing it up.

And so, when you fund these corporate campaigns, you can get big impact. And we believe we came in when there were already a couple wins. There was already some momentum. But we do believe that we were a part of helping to accelerate the corporate campaigns. And fairly shortly after we put in a large amount of money, we saw just this wave of pledges that swept all the fast food companies and all the grocers in the U.S. So hopefully a couple decades from now, you won’t even be able to get eggs that aren’t cage free in the U.S. So the impact there in terms of the number of animals helped per dollar spent already looks like pretty inordinate. And it’s hard to say exactly what our impact was, because the momentum was already there. But I do think of that as sort of an early hit.

And meanwhile, you know, we’re also funding farm animal work in other countries, where things are less far along, because we also wanna seed the ground for those future wins. And we wanna be there for the tougher earlier wins too.

You know, in criminal justice reform, we have seen some effects as well. We picked that cause partly because we thought we could get wins there. It’s another one where it’s been on a slightly shorter timeframe. There was a big bipartisan bill in Illinois that we think will have a really big impact on incarceration in Illinois. There’s a, something that we believe is partly attributable to our funding, and we’re probably going to check it out through our history of philanthropy project at some point, just to kind of check ourselves, and better ourselves.

And in some of our other causes, well, the world hasn’t been wiped out by a pandemic, so that’s good. I mean, some of our global catastrophic risk reduction cause, it’s much harder to point to impact. The nature of the cause is less amenable to that. I think that is some reason that it’s good for that not to be the only kind of cause that we’re in. And we have to [inaudible 01:08:09] more by intermediate stuff. So with AI, we’re happy if we see more people doing AI safety research than used to be doing it, especially if they’re doing it under our funding, which is the case at this time. But it’s an intermediate thing.

Robert Wiblin: What are some other approaches that different foundations take that you think are interesting or that you respect or have some successes in their history?

Holden Karnofsky: Yeah, for sure. We have a certain philosophy of giving, and we’ve kinda tried to emulate a lot of the parts we liked of other foundations. But there’s a lot of really different schools of thought out there that promise to be very effective. So our kind of philosophy is work really hard to pick the causes, work really hard to pick the people, and from there, try to minimize decision makers and emphasize autonomy for the program officers, and kind of have each thing be done by the person who knows the field best, kind of having a lot of freedom. And I think we also take the same attitude toward our grantees, where by default, the best kind of giving we can do is to find someone who’s already great, already quite aligned with what we’re trying to do, and just support them with very few strings attached.

I think you do see some other models that are very interesting, so I think the Bill and Melinda Gates Foundation I think has had a huge amount of impact, and I find a lot of what they do really impressive. And a lot of it has been, I think, in many ways, more kind of driven. It’s been a little bit more, in some sense, top down. So I would say they’ve kind of come up with what they want major decision makers such as governments, to do, such as put more money into highly cost effective things like vaccines. And they’ve just kinda gone out and really gone lobbying for it, or soft lobbying, making the case for it. And I think their work has been a bit more prescriptive. They have their thing they want to happen and they try to push it to happen. They do everything, and they’re very big. But I think a lot of their big wins have looked more like that and less like, let’s say our cage-free work, where we saw something was already going on and we just kinda tried to pour gasoline on it in a sense.

The Sandler Foundation is kind of different in a different direction, and we’ve written about them on our blog. One of the things they do differently from us is they’re just very opportunistic. We try to develop these focused areas that we become very intense about, and we have people who are just obsessed with them and experts with them. And the Sandler Foundation is more like they have a whole bunch of things that they would fund if the right person or the right implementation came along. And so they kind of, they funded Center for American Progress. They funded ProPublica. They kind of were startup in both of these, and Center for Responsible Lending. And a lot of it is like there’s a whole bunch of things they could do at any given time, and they’re not all united by one cause, and they’re just waiting for the right team to come along. And they operate on a very small staff, just like a few people.

So that’s interesting. Then there are foundations that are more kind of operating, so foundations that are kind of doing the work themselves, doing the advocacy themselves. Kaiser Family Foundation, I think, was originally maybe set up as a grantmaker and essentially became a think tank. And I think all those foundations have had really interesting impact.

You know, and then a final one, the Howard Hughes Medical Institute, they’re a science funder and I think they’ve got just this kind of different philosophy from most of what we do, although we do some things like this. But their philosophy is kind of rather than trying to identify where you’re gonna see the most impact in terms of lives improved and kind of back chain from there, a lot of what they do is they just believe in basic science. They believe in breakthrough fundamental science. They believe in better understanding the basic mechanisms of biology, and they have this program that kind of, it’s a program that’s an investigator program that’s setup to identify outstanding scientists and give them a certain amount of funding, and I think it tends to be pretty similar from scientist to scientist. And we do have some work on breakthrough fundamental science, but they have just an incredibly prestigious program. They’ve been early to fund a lot of Nobel laureates, and they really have specialized their entire organization around basic science, which is a little bit of a different frame than most of what we do, although we have some of that going on too.

Robert Wiblin: How do you expect that you’ll be able to change policy, and has that changed since the 2016 election?

Holden Karnofsky: Sure. So I think the way we like to think of it is that we try to empower organizations that can have a productive effect of the policy conversation. There’s a whole bunch of different ways to do that. We have a blog post, an old one called, I think it’s called The Role of Philanthropic Funding in Politics. And something you can do is you can just fund people to develop ideas, new policy ideas, that may be kind of a new way of thinking about an issue. That is different from, for example, funding elections.

You can also fund grassroots advocacies. You can fund people who are organizing around a common population or a common topic, like formerly incarcerated persons. And you can just support these people to organize and to work on issues they’re passionate about and see where that goes. You can also fund sort of think tanks that try to kind of broker agreements or try to take the new ideas that are out there and try to make them more practical.

So I think there’s a whole bunch of different things philanthropists can do, and a lot of the time, the longer term and higher risk, in some ways, the bigger impact you can have. So this book I mentioned before, The Rise of the Conservative Legal Movement, really argues that by focusing more on the upstream intellectual conversation, and that certain funders have a lot of influence on politics that they couldn’t have had if they’d just tried to jump in on an election and start funding candidates, where the candidates kind of already know what they think and what they believe, and you can kind of pick from a very small set of options.

And so I think there’s a whole bunch of ways to fund politics, and what we’ve tried to do is when we pick an issue, we try to pick someone who can see all sides of the issue, and see how all the pieces fit together, and it’s really different for every issue.

Robert Wiblin: Do you think there’s good evidence that trying to change the intellectual milieu, or the ideas that people in the political scene are talking about is more effective than just funding candidates at the point of an election?

Holden Karnofsky: Well, I think it’s one of these things that’s just inherently incredibly hard to measure, because you’re dealing with, I mean, to trace the impact of something like the Conservative Legal Movement over the course of decades on the overall intellectual conversation. You not only can’t do it with randomized controlled trials, it’s real, there’s not much you can do in terms of quantitative analysis period. You kinda just have to be a historian, an ethnographer… And so I think it’s hard to say, and I think a lot of what we try to do is rather than getting everything done to a formula, we try to just be as well informed, as reflective as we can be, and work with the people who are as well informed and as reflective as we can find. And so, when it comes to choosing when are we going for the long term, the big win, versus when are we going for the short term, the tractable thing, a lot of that comes down to once we have a cause, we try to pick the person who we think can think about that question better than we can, and then we follow their lead. And that’s the program officer.

Robert Wiblin: What would you say to a potential philanthropist who wanted to do a lot of good with their money, but their one condition was that it had to be something that Open Phil wouldn’t fund, or Good Ventures wouldn’t fund?

Holden Karnofsky: Sure. So, I mean, I’m kind of a strange person to ask about this, but I do have some answers none the less. First off, there are things we just don’t fully fund for various reasons. We don’t fully fund GiveWell top charities. That’s for reasons that have been laid out in great detail in the cause prioritization post. There are organizations where we don’t wanna be too much of their funding. We put out an annual post on suggestions for individual donors, so that’s a possibility. And then, the other thing is, there are these kind of, there’s the EA funds, some of which are run by people who work in Open Philanthropy. And those are more specifically to give people who work here the opportunity to fund things that Open Philanthropy can’t or won’t do. And there is some stuff in that category. And so occasionally, we’ll be looking at a giving opportunity, and we’ll say, that poses too much of a communications or a PR risk, or there’s some other reason, or we don’t have full agreement among the people who need to sign off on grants, which we try to minimize the number of veto points, but sometimes we still won’t have agreement.

So an example of something is, something that I am somewhat interested in as a philanthropist is the idea of experimenting with just like experimental individual re-granting, so just taking someone who I think it would be interesting if they had a chance to try their own philanthropy and see if they would do it differently from me. And we might have enough in common in terms of values and in terms of worldview and in terms of goals, that I can feel confident that if we sort of re-grant, just granted a bunch of money to them to re-grant, that I could feel comfortable that money would not be used terribly or the person would work really hard to optimize it, and they might find something much better than what we could do. And if they couldn’t, they might give it back. And so that’s something that we’ve talked about experimenting with, but I think it’s, I think there are some logistical obstacles and there’s also some just internal … It’s not just, I think there’s a higher level of excitement from me than from some of the other people, such as Cari and Dustin, for this idea, and so we aren’t necessarily experimenting with it at the level or pace that I would necessarily do if it was my money.

And I think that’s a pretty interesting idea is just look around yourself, because there’s someone I know who they really might want a shot at being a philanthropist. Who know what they would come up with and they have a lot in common with my values. That’s something that I’d be pretty interested in and that I’ve sometimes encouraged people to try out.

Robert Wiblin: And one of the big benefits there is that they can just use kinda the unique knowledge that they have that’s very hard to communicate.

Holden Karnofsky: Exactly.

Robert Wiblin: Yeah.

Holden Karnofsky: Yeah, exactly. And one way of putting it is like, you know, if you gave, if you have … Let’s say, just to use some made up numbers, let’s say you had a billion dollars and you gave a million to someone who you wanted to see what they would do. The idea would be like their first million dollars is better than your last million, because it’s like they’re, in some sense, their ration of human capital to money spent is gonna be extra high for that money, and so they might put extra time into finding things that you don’t have time to chase down, ’cause you can only run so big of an organization without starting to run into organization issues that we don’t necessarily want to contend with yet.

So I think that’s part of the thinking. And yeah, also part of the thinking is this is a theme running through Op