Transcript

Robert Wiblin: Hi listeners, this is the 80,000 Hours Podcast, where each week we have an unusually in-depth conversation about the world’s most pressing problems and how you can use your career to solve them. I’m Rob Wiblin, Director of Research at 80,000 Hours.

Today’s episode is going to be incredibly useful for anyone who has considered working on the AI alignment problem, or artificial intelligence or machine learning more generally.

It also explains ways to greatly speed up your career and sidestep doing a PhD, which I expect can be generalised to some other areas as well.

It comes with a written step-by-step guide written by Catherine Olsson explaining how you can transfer into a research engineering role in machine learning as soon as possible. We’ll link to that in the show notes and blog post attached to the show.

If you’d like to follow that guide and retrain to get an ML engineering position, but don’t have the cash in the bank to pay your living expenses while you do so, the Effective Altruism Grants program may be able to fund you whenever it next opens. We’ll stick up a link to their site as well.

If you have limited interest in AI, ML, PhDs or software engineering you may find this episode goes too far in the weeds for you.

But I expect you’d nonetheless be interested in earlier episodes that introduced this topic like #3 – Dr Dario Amodei on OpenAI and how AI will change the world for good and ill, #31 – Prof Dafoe on defusing the political & economic risks posed by existing AI capabilities and #44 – Dr Paul Christiano on how we’ll hand the future off to AI, & solving the alignment problem.

I also have a really important appeal to make to you all. We’re getting close to the end of the year, which is when 80,000 Hours tries to figure out whether all the things we’ve been working on throughout the year have actually been of use to anyone.

We need to hear from people how we’ve helped them so that we know we just shouldn’t give up and get different jobs. And our donors need to know we’ve changed people careers so they know to keep funding us. If we couldn’t find out those stories, 80,000 Hours wouldn’t exist.

So if anything 80,000 Hours has done, including this podcast, our website or our coaching has changed your career plans or otherwise helped you, please go to 80000hours.org/survey and take a few minutes to let us know how.

That’s 80000hours.org/survey.

OK here’s Daniel and Catherine.

Robert Wiblin: Today I’m speaking with Daniel Ziegler and Catherine Olsson. Daniel and Catherine both studied at MIT, with Daniel majoring in computer science and Catherine double majoring in computer science and brain and cognitive science. Daniel started an ML PhD at Stanford, but left and is now a research engineer at OpenAI. Catherine started a computational neuroscience PhD and then left with a master’s degree to join OpenAI as a software engineer. And now she’s at Google Brain as a research software engineer. Thanks for coming on the podcast, both of you.

Catherine Olsson: Thanks so much.

Daniel Ziegler: Yeah, excited to be here.

Robert Wiblin: So in today’s episode we hope to get more information about how technically inclined people can move their careers into the direction of aligning AI with human interest. But first, can you each describe what you’re doing now and why you think it’s important work? Maybe Catherine first.

Catherine Olsson: Sure. So right now at Google Brain, I’m a research software engineer as you said. I’m working on Ian Goodfellow’s team, which works on adversarial machine learning. And that includes situations where there’s a real adversary that’s trying to cause your machine learning model to make a mistake, or there’s like a contrived adversary as part of the training process, as in a generative adversarial network that’s part of the process of making the network learn better.

Daniel Ziegler: Yeah. So I work as a research engineer on the safety team at OpenAI, and basically, my job is to take ideas that some of the researchers there have for training aligned AI systems and implementing concrete prototypes and experimenting with those prototypes and seeing what actually matters and how we can make them work better. We’re working on a bunch of techniques for getting a whole bunch of human feedback and using that to train AI systems to do the right thing.

Robert Wiblin: How do you think those products will ultimately contribute to AI safety in the broader picture?

Catherine Olsson: From my perspective, many paths to powerful AI systems will involve machine learning. I think many folks would hope that we can just construct by hand a system that we knew exactly how every piece works, but that’s not how the current deep learning paradigm is proceeding. Where you throw the system at a lot of data, it learns whatever it learns, and the thing that is has learned in current systems and in systems we can foresee, is not exactly what we wish it would have learned.

Catherine Olsson: So current systems like classifiers are the ones that adversarial examples are typically demonstrated on, where you can make very small perturbations to the image and get the classifier to produce a completely different label than what a human would put on that image. So this is just evidence that the systems that we have today are not learning the types of concepts that humans would hope that they learn, and they’re not learning robust concepts. If systems that are anything like this are gonna be in powerful systems, we will have to iron out these issues first. That’s one narrative I have for how this sort of work might contribute.

Catherine Olsson: The sort of problems that we’re working on in a research sense in this field are still ironing early kinks in the process. It’s very unlikely that any real world system is gonna get thrown off by someone going and changing every pixel a tiny amount. But it is a real valuable toy problem for us to be honing our formalisms and our abstractions in this sort of a research domain.’

Daniel Ziegler: Yeah. I like to frame the problem of deploying a powerful ML system in a way that’s actually safe and actually beneficial like… we can kind of break it up into two parts, where the first part is just giving it the right objective at all. So that’s like optimizing for the right thing. And the second part is trying to optimize for that objective robustly. And optimizing for that robustly includes things like being robust to adversarial examples, or optimization pressure from other parts of the system. It includes things like being safe during the training process, like with safe exploration in reinforcement learning.

Daniel Ziegler: We’re definitely really interested in all those kinds of robustness things, but so far on the safety team, an even more central focus has been this thing about giving our AI systems the right objective. We’re definitely not very optimistic about trying to write down some perfect utility function for our AI systems or some perfect reward function for our AI systems directly, right? It seems like there’s too many things that we care about and whatever we try to specify directly for interesting tasks is gonna have loopholes in it that are gonna be exploited by powerful systems.

Daniel Ziegler: Our plan, really, is to come up with mechanisms for collecting a whole bunch of human demonstrations and human feedback and using them in a way to direct the training process of a powerful AI system to do the right thing. If we define the right objective and we train for it in a robust fashion, really we should be done. We should have a system which does what people want and so in some sense, that’s the whole problem. Maybe it’s kind of a broad view of safety, of saying if you have a powerful system that’s acting in the world and has a lot of influence, for that system to act safely, it’s going to need to understand quite a bit about human values.

Daniel Ziegler: You know, maybe it doesn’t have to have a perfect understanding, maybe sometimes it can act sort of cautiously, but it’s going to need to understand a lot and we want to do that as well as we can.

Robert Wiblin: Yeah, so listeners might know a little bit about OpenAI already, cause we spoke to Dario Amodei last year. But I imagine things have shifted a bit cause the organization’s only a couple of years old and it’s a pretty fast moving space. So what’s the broader picture of what OpenAI is doing?

Daniel Ziegler: Yeah, I mean, I’m pretty new to the organization. I’ve only been here for about four months, but OpenAI’s got a lot of different projects going on. There’s a lot of individual teams. Some of them are teams focused on one big project like the Dota team developing OpenAI Five, which has recently had some exciting matches. There’s the OpenAI robotics team, which recently had a release with Dactyl, a robotic hand. But then there’s also other teams like the multi agent team or the safety team, which have taken more of a portfolio approach, where there’s a bunch of smaller projects that one to two researchers are working on.

Daniel Ziegler: Most of OpenAI is really focused on doing a bunch of research and a bunch of large scale engineering projects to try to push forward AI. And the safety team is trying to harness that and harness other advances in the field, and trying to stay at the cutting edge there and apply those techniques to systems to make aligned AI.

Robert Wiblin: Yeah. And on the other hand, I know remarkably little about Google Brain. How’s it different from DeepMind and has there ever been a consideration of putting them together? Is there a difference of focus and what’s Brain’s main projects?

Catherine Olsson: Google Brain is part of Google, which is under the Alphabet umbrella. DeepMind is also under Alphabet, but they’re separate in many ways, and there’s a lot of collaboration as well. DeepMind’s motto is something along the lines of “Solve intelligence, use it to solve everything else.” Brain does not have quite a AGI or general intelligence focus, and you can see that in the types of research that’s being done. The work that I’ve liked that’s come out of Brain includes machine translation systems, for example, which are being deployed in things that we’re seeing today.

Daniel Ziegler: Yeah, and OpenAI’s mission is more AGI focused like DeepMind, so OpenAI’s mission explicitly is to build safe artificial general intelligence. I’m really excited that safety is part of the mission statement there.

Robert Wiblin: Yeah. So Brain is more focused on specific products that Google can develop, like the translation.

Catherine Olsson: Well, the translation where it’s basic research. Brain is not a product focused team, but it’s basic research that more holistically covers the space of things that machine learning could be applied to as opposed to specific general purpose agents, which I think is a more narrow focus that places like OpenAI and DeepMind are focusing. The things that Brain is producing is not focused on general purpose agents of this highly specific kind. It’s anything that sort of uses machine learning or learning or representations to produce useful stuff.

Catherine Olsson: Not product per se, but basic research that more spans the space of types of tasks that machine learning can be applied to.

Robert Wiblin: Can you say a bit of why you decided to work on artificial intelligence?

Catherine Olsson: For me it’s sort of an interesting story that when I was in my PhD program, I… for all of my life wanted to study something at the intersection of minds and machines. I thought systems that process information are very interesting, and we’ve got some inside our heads and some that we can program. Something in this space would be really cool to work on, and so I joined the PhD program because I wasn’t done with these questions yet, and I was very excited about the lab that I joined and the work going on in that department. But I found it kind of solitary and kind of slow paced. And so I was looking to go back in the software engineering direction towards the sort of stuff I’d done in undergrad with the computer science degree that I’d finished.

Catherine Olsson: I actually expected that I would have to go with something way more boring to get back to software engineering. I was like, “Oh, I’m gonna go do data science at a company I don’t care about, but at least it’ll be more collaborative and faster paced and I’ll be happier.” Then I reached out to Greg Brockman, actually, who also went to MIT. And I knew him not particularly well, but well enough that I could say, “Hey I hear you’ve got this company called OpenAI. Do you have any jobs?” And he was like, “Actually yes. We’re hiring software engineers.” So it was really fantastic timing for me.

Catherine Olsson: At that point I was thinking that machine learning or deep learning, particularly, was on the rise, and so it was a good time to jump on that bandwagon, just for ordinary reasons of career prestige and high pay. You know, I had a lot of friends who were concerned about long term impacts from artificial intelligence and I’m like, “Well, let me just go to a place where people are talking about this and just sort of see what’s up.” And you know, if I’m gonna work in this space, I’d rather work in a place that, as Daniel said, has the beneficial impacts of humanity top of mind.

Catherine Olsson: My impression after ending up at OpenAI is that there’s really just not that many people thinking extremely seriously about the long term. That sort of inspired me to keep that on my radar as one of the things that I can be steering my career towards.

Daniel Ziegler: Yeah, I would say I really came to AI and AI safety in particular through effective altruism. I had been doing computer science all my life and was studying computer science in undergrad, and when it came towards the end of my undergrad and I was thinking what kinds of careers could I actually use this for that would actually have an impact in the world… You know, I’d been hearing more and more about AI safety over the years and just became increasingly convinced that AI was gonna be a big deal, very plausibly gonna have a huge impact on the world one way or another in the not that distant future. So it seemed like a natural transition.

Daniel Ziegler: I mean, I didn’t have that much ML experience, but I did have a lot of computer science background in general, so it seemed like it should be a very manageable jump, so I decided to try to go for it.

Robert Wiblin: So what’s the path that took you to working at OpenAI? I guess most people who graduate with computer science say… You know, I wouldn’t imagine that they could quickly go into a job at OpenAI, but you managed to do it.

Daniel Ziegler: Yeah, so the funny thing is that originally I decided that I would try to get an ML PhD, to try to do AI safety research in academia. So I applied to a bunch of top PhD programs in ML and I got into Stanford despite the fact that I didn’t really have any ML experience. I did do some pretty strong undergrad research, but that was in formal verification, so like I helped prove a file system correct, proved that it will never lose your data and stuff. But that really had nothing to do with machine learning, so it was definitely a pretty big leap.

Daniel Ziegler: I sort of figured hey, if they accepted me, might as well give it a shot. Yeah, and when I ended up at Stanford, there were a couple of things that came together that actually made it so that I went on leave after just two weeks there. What happened was like, I was sort of starting to have second thoughts anyway, because I just noticed that my ML background really was a bit lacking and I would have to do a lot of catch-up. That was gonna be pretty hard work to get to the point to where I could actually do useful research. Maybe more importantly, I just noticed that I didn’t really have my own vision for how to do AI safety work, AI safety research that I thought was actually gonna be useful.

Daniel Ziegler: I definitely thought I was gonna be able to find incremental things that definitely rhymed with AI safety and seemed kind of useful, but nothing that I thought was really gonna make a big difference in the end. And on top of all that, I actually got a brief medical scare where I downloaded 23andMe’s raw data and uploaded it to another service, which was not the smartest… uploaded it to another service to analyze the data, and it told me that I have this rare genetic heart condition, hypertrophic cardiomyopathy. That was literally right as I was starting my PhD at Stanford, and already stressed out about the situation.

Daniel Ziegler: In hindsight, I took it far more seriously than I should have. It was pretty likely to be a false alarm, and after a bunch of tests, I figured out, yes, it was a false alarm. But at the time, with all this coming together, I was like, alright. I’m gonna step back and take some time to reevaluate, and so that’s what I did.

Robert Wiblin: I guess this is a huge aside, but do you have any advice for people that are doing 23andMe at this point? I mean, I’ve considered doing it, but this is one of the reservations I’ve had, that I’m gonna discover lots of terrible things that are in my future and that I’m not gonna be able to do anything about.

Daniel Ziegler: Yeah, yeah, yeah. Well, I think you can do it responsibly. 23andMe lets you choose whether you want to even see some of the more life impacting things, like whether you’re at high risk for Alzheimer’s or whatever. You can just turn that off and not even see. Also, I went another step beyond that and downloaded all of the raw data and uploaded to another service, without any FDA approval or anything for any of those tests. And the thing to remember there is it’s like a huge multiple hypothesis testing problem. Because you have thousands of SNPs, single nucleotide polymorphisms, and each one of them maybe have a 99.9 plus percent accuracy. But of thousands of them, you’re pretty likely to get a false positive somewhere that says you have something terrible.

Daniel Ziegler: So if you do take the step of downloading the raw data that way, you should think it’s only a very slight… and if something comes up, there’s only a very slight suggestion that something is actually wrong. Maybe it’s worth following up on, but definitely you should not take it that seriously.

Robert Wiblin: Yeah. Okay, so returning to the PhD. I know quite a lot of people who have applied to do ML PhDs. Sounds like you went into it with perhaps not quite enough background in ML, like it was going to be a lot of work to keep up. Do you think… does that generalize to other people? Should other people try to do masters or some other study first?

Daniel Ziegler: Yeah. I mean, I think that probably it would be smart to start a few years earlier and… to get into a top PhD program, you kind of need to have done some research already, and gotten some publications, or at least done research that’s got you really good letters of recommendation from your advisors. And it would be a lot more convenient if that was just in machine learning already. So if you have that head start and can start doing machine learning research right away, that’s definitely gonna be a much easier path.

Daniel Ziegler: I think what I did is certainly doable. People do make those kinds of transitions. But you should definitely expect it to be a little bit more difficult.

Catherine Olsson: Can I shove in with some more PhD rants?

Robert Wiblin: Definitely.

Catherine Olsson: There’s a lot of advice about doing PhDs out there. I have just some meta advice, which is like… that advice often just comes without reasoning. It’s like a barrage of like, read lots of papers! Also talk to lots of people. Make sure that you’re publishing, but only in the good conferences. Pick a really good advisor, you know, who have all the good advisor things about them. There’s a lot of advice that’s not really grounded in any particular goal, and when I turned up to do a PhD, I was just trying to follow all of the advice in a kind of undirected way. I just wanna study stuff in this area.

Catherine Olsson: I now think a PhD is gonna go a lot better if you have a pretty clear goal, even if it changes, right? What’s the saying? Plans are useless, planning is essential? I think I’ve mangled that a little bit, but come in with something you’re trying to do.

Catherine Olsson: A PhD is a little like a hacker space, where if you’ve got a project in mind, even if it’s kind of a toy project that you’re gonna throw out, you can come in and start using those resources towards something. Whereas if you turn up at hacker space and just start plugging stuff together, you’re not gonna produce anything. And PhDs in general are often particularly tough if you don’t have experience with yourself of how to motivate yourself in an unstructured environment. I don’t tend to think of that as like, oh, there’s people who can do it and people who can’t. It’s like, have you got the life experience under your belt, whatever it takes for you to know how you work best when you aren’t given that structure by someone else.

Catherine Olsson: I would recommend that folks consider doing something other than going straight from undergrad to PhD, because the undergrad experience is so scaffolded and structured where you’re assigned tasks, and the PhD experience is so unscaffolded and unstructured that anything you can do to give yourself some time to feel out, “What am I like when I’m working on something where no one has told me what the steps are?” And feel that out in other, lower stakes contexts, whether that’s side projects or a job where your manager lets you spend 20% of your time on less structured stuff.

Catherine Olsson: Then once you feel like you’ve got your own motivational house in order, then to jump into something as motivationally fraught as a PhD, you’re more likely to have the experience that you need for your own motivational scaffolding.

Robert Wiblin: Yeah. So you left after two years, was it?

Catherine Olsson: Three years, yeah. I would’ve left after two but unlike Stanford, NYU is not as lenient with taking leaves, and so I spent a whole other year trying to make it work for me, which… it didn’t, but I tried.

Robert Wiblin: Yeah, do you wanna talk about that?

Catherine Olsson: Yeah. So, as I said, my experience coming in was that there topics that I found interesting and wanted to work on, but didn’t have a specific goal of like, “I wanna discover this thing about the brain works in order to achieve this outcome.” It’s interesting. For a long time, I was unhappy, but I didn’t know why. Everything on the surface seemed to check out, like, good lab, good project, got the skills that I need. Why am I unhappy? And ultimately I realized that I’m allowed to just leave cause I’m unhappy. I don’t need to know why.

Catherine Olsson: I do now have hypotheses that I think are pretty solid. As I said, for example, comparatively solitary work and comparatively slow feedback cycles, which led me to something like software engineering which is more group or team based a lot of the time and has faster feedback cycles. But also I think not having a particular thing in the world that I was trying to move or do or cause, I think, led me to feel ultimately, what of this is gonna matter? And the pure raw curiosity wasn’t driving me as much. And I think some folks can succeed motivated just off the scientific curiosity of, “How does this work? I just wanna figure it out.” But for me, that has to be just one piece of the large puzzle.

Daniel Ziegler: Yeah, and I think I sort of was afraid that something kind of similar was gonna happen to me. Where I sort of wanted to work on AI safety in the abstract, but I didn’t have a particular research agenda that I had come up with that I was really burning to try to execute on and specific questions that I really wanted to understand. So without that, I think it would’ve taken a while until I had actually found something that I thought was valuable.

Catherine Olsson: So can I get on a soapbox about that? Like, one of my favorite soapboxes to get on, but I’m gonna… I’m gonna get on.

Daniel Ziegler: Okay.

Robert Wiblin: Love soapboxes on this show, so go for it.

Catherine Olsson: So the thing you’re pointing out is that you wanted to work on AI safety in the abstract, but you didn’t have a particular question that you wanted to work on. I see this a lot in people that I talk to that are interested in AI safety, and my sort of party line is that “AI safety” is not one thing. It’s definitely not one field. If anything, it’s a community of people who like to use the phrase “AI safety” to describe what they’re interested in. But if you look at what different groups or different people are working on, they’re very very different fields of endeavor. So you have groups that are trying to take deep reinforcement learning and introduce a human feedback element so that you can learn human preferences in a deep RL system. That’s one research agenda. There are many ways that that could end up in a future system.

Catherine Olsson: Another agenda is, for example, MIRI has folks working on decision theory. If we understood decision theory better, than we would know better what a good system should be like. Okay, decision theory theorem proving is just categorically a completely different type of work from deep reinforcement learning. You’ve got groups that are working on… so like my group, for example, working on robustness in machine learning systems. How do we know that they’ve learned the thing that we wanted them to learn? Also a completely different field of endeavor.

Catherine Olsson: And it’s very important to keep in mind if you’re looking for a, quote, career in AI safety, is what exactly is it that you think is gonna be important for what trajectory that you think the world is gonna be on? And then what are the particular subskills that it’s gonna take? Because it’s not a monolith at all. There’s many many different groups taking many many different approaches, and the skills you need are gonna be extraordinarily different depending on the path.

Daniel Ziegler: Yeah. And one thing I wanna say, and I wonder if you agree with this, is to some extent, it is possible to off-load your… especially if you’re going for more of a research engineering position, it is possible to off-load some of your opinions onto other people. Like I sort of looked out at all the different people doing different kinds of AI safety work, and basically, my call was that what the OpenAI safety team was doing was to me seemed like the most promising and the most valuable approach. And I did have to make that decision myself, and understand what I thought the trade-offs were between different approaches, but I didn’t have to come up with the approach myself. And I was able to piggyback on other people’s thinking about this space, and join this team with an existing research agenda that I could just help out with. That, I think, was really useful. That said, the more you can have your own opinions and your own ideas about AI safety, the better, for sure.

Catherine Olsson: I’ll answer your question, but I wanna push you a bit.

Daniel Ziegler: Uh-huh.

Catherine Olsson: Approach to what? You thought it was the best approach to what, exactly?

Daniel Ziegler: Yeah, I think that’s absolutely the right question to ask. I think it’s definitely a mistake to come in from the EA space and be like, “Oh, AI safety, there’s this thing.” Which I think is exactly what you’re pointing at. So to try to be more specific about how I think of what I’m trying to solve, is basically, I expect that at some point in the future… and it could be in not all that much time, we’ll have powerful AI systems that exceed human performance on many kinds of cognitive tasks.

Catherine Olsson: And sorry, you mean some sort of autonomous agent-like system?

Daniel Ziegler: So it doesn’t necessarily have to be agent-like systems. It could be systems that are just recommending steps for humans to take or whatever. But they might be systems that are taking in a huge amount of information or doing a whole bunch of reasoning on their own in a way that’s really difficult for humans to oversee. And I think there’s gonna be really powerful economic incentives and strategic incentives for various actors to deploy powerful systems like this.

Daniel Ziegler: So I don’t want to end up in a world where there’s this vacuum where there’s this powerful incentive to deploy systems that are used to help make important decisions or maybe are acting autonomously themselves, but we don’t know exactly how to specify what they should do in an effective way. So that’s really why I’m super excited to work on some of the stuff that the OpenAI safety team’s working on, which I would definitely wanna explain a little more about.

Robert Wiblin: Almost the core of the conversation is that the path from doing a PhD or finishing a PhD to then going into one of these roles is pretty clear and we’ve got lots of coverage of that. But I think people are a lot less clear about how do you get into really useful research roles in AI safety broadly construed without finishing a PhD?

Robert Wiblin: So you left the PhD and then what happened next? What did you decide to do and why?

Daniel Ziegler: I basically just spent a couple of months taking some time off and thinking about what I wanted to do. I did a couple of little projects, like I helped out a little bit with a research project at CHAI, the Center for Human-compatible AI at Berkeley. I thought a little bit about how could you try to use prediction markets to try to understand what’s gonna happen with AI. But yeah, then I decided, I think, that like… I’ve always enjoyed software engineering and I like building stuff, so I thought that trying to work in a research engineering position for an AI safety org could be a really good path for me. So I decided to apply to OpenAI and to MIRI, and I spent basically a solid month and a half just preparing for the OpenAI interview.

Daniel Ziegler: What I did was, Josh Achiam who was one of the researchers on the OpenAI safety team, has a list of fifty or so deep reinforcement learning papers, like the key papers in deep RL. And I just went through this list together with a housemate of mine, and we read one or two papers a day, and picked a handful of them to implement and to actually try to reproduce. So I spent a bunch of time coding in Python and TensorFlow, coding up these deep reinforcement learning papers and trying to debug things and trying to tune things until they were actually working reasonably well.

Daniel Ziegler: I think generally the advice is like, if you’re trying to get good at something, just do that thing and then see what’s necessary to be able to do well at that. So I just jumped into that, and then I applied to OpenAI, got that job. I also went through MIRI’s interview process, and they did end up giving me the offer as well. And then I spent some time trying to decide and ended up thinking OpenAI was the place to go.

Robert Wiblin: So it seems like there was a huge potential shock out here, cause you could’ve spent, what, four to seven years doing the PhD –

Daniel Ziegler: Absolutely.

Robert Wiblin: And instead you spent six weeks reading papers. What do you make of that?

Daniel Ziegler: Yeah, yeah. Certainly, I think I would’ve hoped to do useful research, AI safety research, during my PhD as well. I think that if I hadn’t been able to do that, I would’ve been pretty unhappy spending all that time just sort of in preparation for hopefully doing something useful. Even so, I think this was a much more effective path than I could’ve taken otherwise.

Catherine Olsson: I’ll emphasize that I also see that the path might be shorter than you think as a theme, at least right now, in 2018, in the landscape, where as I mentioned there’s many different corners or different approaches or groups working on something that could be construed as AI safety or could be relevant to AI safety. And some of these corners, like Dario’s team at OpenAI, are just hiring engineers. Similarly, MIRI is just hiring straight up software engineers. Now, it’s not the case that every problem that needs to get solved in order to deploy powerful systems safely is at that point, but some of them are. And so if you are a talented software engineer and you’d like to start working on it, the state of the questions right now is that some of them are just ready to throw engineers on. And so if you haven’t just tried applying to the position that you want, just try. Just see. You might actually be ready for it.

Catherine Olsson: There are many total unsolved wide open pre-paradigm questions that you can’t just throw a software engineer on. And those need a different sort of approach, so it’s valuable for folks who think these questions are important to consider the full spectrum of problems that we need to solve and see which of those can I best contribute to. And some of them, we can just throw software engineers on, and if that’s what you wanna do, go do that. And some of them need highly productive researcher effort to crack open pre-paradigm corners of the space, and that’s just a very different kind of thinking. It’s a very different kind of work. But all these kinds are currently useful.

Robert Wiblin: Yeah. What are some of the problems that are ready for software engineers?

Daniel Ziegler: Yeah, so I mean, on the OpenAI safety team, we just decided to open up a position for just a generalist software engineer that doesn’t necessarily have to have any ML experience. And the idea there is that we’re getting big enough now, we’re like six people, and with a couple of new people joining us soon, that we do have a bunch of tasks where a software engineer can help us a lot. Things like making a website to collect human training data or optimizing various parts of our machine learning workflow to make things run faster, to manage all the experiments that we’re running. So basically both things that are directly part of the experiments and things that will help on the workflow for the researchers and the research engineers. And that’ll all be very useful.

Robert Wiblin: Catherine, it sounded like you had some other things in mind as well?

Catherine Olsson: Yeah, so I was also gonna point out that in the field of adversarial examples, which is in my opinion a research field that has a current paradigm… and I sort of mean paradigm in the Kuhnian “Structure of Scientific Revolutions” sense, where there’s a set of questions and there’s a set of tools, and there’s a community of researchers that agree that if you apply those tools to those questions then you yield something that we all recognize as progress. So there’s a crank to turn, and there are many many researchers who are now working on adversarial examples turning that crank.

Catherine Olsson: And so that’s a case where you don’t have to necessarily know how to go into a totally uncharted research field and find a foothold in the middle of nothing. If you’re a competent researcher or research engineer, there’s work you can do right now on attacks and defenses in the current adversarial examples paradigm.

Catherine Olsson: And I think that’s not quite just ordinary software engineer. That’s like, if you know TensorFlow and you can read your way through papers, you don’t have to be like at a PI Principal Investigator level, like got your PhD researcher. You can be early stage PhD and sort of jump on some of these open problems in adversarial examples and sort of push that forward.

Catherine Olsson: Now in my view, I think that field is gonna need to move on to the next paradigm, right? Like, the “make small changes to images” paradigm is currently still fruitful, but it’s starting to sort of show cracks in that paradigm. That’s not a particularly realistic threat model in the real world. What could we move to that’s a better threat model? Or the theory of adversarial examples, also just starting to get opened up.

Catherine Olsson: And so I think this is a regime where if you’re a research engineer or early stage researcher, there are these sort of juicy hand holds that’ll help us move this paradigm into the next phase or the next paradigm to come, where we’re starting to really understand what’s going on with these problems and then how that might relate to systems that actually get deployed in the real world.

Robert Wiblin: So just to back up, you left your PhD, and then what happened?

Catherine Olsson: Right. So as I was considering leaving, that was the spring of 2016. That’s when I reached out to OpenAI, among other places I’d reached out to, and said like, “Hey, do you have a job for me?” And OpenAI said, “Oh actually, here, do a trial with us. Spend your afternoons and evenings, throw together this API for this tool that we’ve built.” And that was a really fun way for me to just see what that work would be like. And that went well and I said like, “Okay, well I’m about to go on vacation, but when I get back, happy to do more trials.”

Catherine Olsson: So I moved to San Francisco with the hope that that would work out, but also sort of considering other job opportunities. And that did end up working out and I was able to jump over to OpenAI then. But it was a bit of a leap of faith, like, “Well, I’m gonna go to San Francisco. There’s a lot of jobs that I like out there and one of them I’ve already had a successful initial trial, and there’s other opportunities I could consider as well.”

Robert Wiblin: Yeah. So I guess for both of you, what do you think it was that made OpenAI interested in hiring you?

Daniel Ziegler: I think for me it was mostly just, you know, doing really well in the interview. I mean, I think, like once you get your foot in the door, once you can convince them that you’re reasonably likely to be capable of doing their kind of work based on your background. It’s really just a matter of demonstrating in the interviews that you’re able to quickly code a bunch of machine-learning, reinforcement-learning code and get it to work and do the kind of stuff that you’re going to do day to day on the job.

Catherine Olsson: Different organizations are looking for very different skill sets or personality types. OpenAI, when I joined, was very small, I was really excited about jumping in on the ground floor of something that was still forming itself, and I think I brought that demeanor to the table as well as just the technical chops of like, yep, you can throw me in on coding something that I’ve never seen before and I’ll be able to do it. I think, for any given organization that you might be looking at applying to, great questions to ask, and this is generic career advice, but ask the questions like, what’s valued here? What sort of person thrives here? It’s going to be very different for different places. Now that I’m at Google, I feel like it’s a different set of skills that I’m bringing to the table. It’s not like a totally new, changing culture. It’s actually quite an established culture, but now the types of skills that I’m bringing to the table include working on a team to figure out what this team needs. What project could I pick up to fill in the gap here? What skills can I develop that are underrepresented among this group of eight to 10 of us who are working on similar problems.

Robert Wiblin: So, you work on that computer game thing, I’m guessing, at the time, in 2016-17 at OpenAI?

Catherine Olsson: Yes. I was working on the Universe Project.

Robert Wiblin: That’s it.

Catherine Olsson: Exactly.

Robert Wiblin: It’s not called a computer game.

Catherine Olsson: Well, then I worked on the other computer game thing, namely the DotA 2 project.

Robert Wiblin: And that was wound down, I think, in 2017, because… the first one, Universe, people just found out it was too hard and they thought, oh, well, we’ll have to do this…

Catherine Olsson: Yes. So Universe was a project to allow any game that you can run in a Docker container, like any game that you can run on a computer basically, to be an environment for reinforcement learning research. It was a highly ambitious goal. The hope was to unlock many more environments. The quantity of the environments would drive research progress.

Catherine Olsson: In my take of why it didn’t ultimately get taken up as a set of environments by researchers, one is that they ran in real time. The ability to run faster than real time is currently a major driver of progress in reinforcement learning. Another is the environments were not all that high quality, and so you don’t necessarily know exactly what training signal is my agent being fed, and the current work was through operating in a small number of environments that researchers know very intimately, which was not the vision of that project.

Catherine Olsson: I still think something like that is a really useful target for transfer learning or generalization in agents, but it wasn’t the correct next step for the research community to be tackling.

Robert Wiblin: Okay, so people have moved on to DotA 2. This is Defense of the Ancients.

Catherine Olsson: Yes.

Robert Wiblin: It’s a computer game that AI’s are learning to play.

Catherine Olsson: Right. So, DotA 2 is a very popular multiplayer computer game, five players on a side. It requires a lot of teamwork, communication, long-range strategy, et cetera. When I joined that project, it was first starting up. I was working on the evaluation system. So how do we know how good this agent is? We came up with a system for playing the agents against one another, ranking them so you can see the progress over time. It’s a multi-agent, or player v. player setup, so you don’t have just one score, like how good is my classifier or how good is my agent against the level. Your agent is always tied against itself as it’s doing self-play. So how to turn, while it’s just still tied, into a meaningful measure of progress involved loading past agents, future agents, playing them against one another, and then looking at how that win rate changed over time.

Robert Wiblin: Okay, so, Daniel, now that you’re working at OpenAI, what is the main agenda there that you’re contributing to?

Daniel Ziegler: Yes, so maybe I will start by describing some of the past work that the safety team did when I wasn’t there yet. There was a NIPS paper last year called, Deep Reinforcement Learning from Human Preferences. The idea there was, normally in the reinforcement learning paradigm, you have some agent acting in some environment, so it might be playing a video game or it might be controlling a robot, it may be a real robot, maybe a simulated robot, and it’s trying to achieve some sort of well-defined goal that’s assumed to be specified as part of the environment. So in a video game, that might be the score. In robotics tasks, it might be something like run as far as you can in 10 seconds, and something that’s a hard-coded function that’s easily specified as part of the environment.

Daniel Ziegler: For a lot of more interesting, real-world applications, that’s not really going to work. It’s too difficult to just write down the reward function that tells you exactly how well you’re doing, because there’s just too many things to take into account. The Safety Team said, okay, let’s relax this assumption and instead of assuming that the reward function is built into the environment, we’ll actually try to learn the reward function based on human feedback.

Daniel Ziegler: So in one of the environments, one of the environments that was like a little simulated robotics task where you have this little hopping agent, just like a big leg basically, we gave humans these examples of what the leg was currently doing. We gave them two examples, one on the left and one on the right, and then the human had to decide which of those was doing a better job, according to whatever the human thought the job should be. So one thing we got the little hopper to do is to do a back flip. It turns out, it’s actually pretty tricky to write down a hard-coded reward function for how to do a back flip, but if you just a few hundred times show a human, is this a better back flip or is this a better back flip, and then have the system learn from that what the human is trying to aim for, that actually works a lot better.

Daniel Ziegler: So the idea is, now instead of having to write down a hard-coded reward function, we can just learn that from human oversight. So now what we’re trying to do is take that idea and take some other kinds of bigger mechanisms for learning from human feedback and apply real, natural language to that. So we’re building agents which can speak in natural language themselves and maybe take natural language feedback, and trying to scale those up and move in the direction of solving more real tasks.

Daniel Ziegler: In the past, we’ve just solved all these really small tasks, but they really are toy tasks. We don’t actually care about making agents do back flips. We care about solving actually interesting problems and working with natural language is a step on the way to that.

Robert Wiblin: It’s understanding feedback that people say? What do you mean?

Daniel Ziegler: Yes.

Robert Wiblin: That sounds pretty wild.

Daniel Ziegler: Yes. I mean, obviously, we’re not at the point where it really works with full human natural language and actually understands what you’re saying, although; natural language models in machine learning are starting to get surprisingly good. You can train them to the point where they are saying things which sound like pretty reasonable text. It’s sort of an ambitious step to try to start working with those things and see how well we can make our ideas, or our alignment ideas, work with natural language. The sooner we start, the more time we’ll have to make that work. We basically want to be on the cutting edge as soon as we can.

Robert Wiblin: So at the moment…

Daniel Ziegler: You know, we’re not, we’re definitely starting pretty simple. We don’t have anything really groundbreaking yet, but that’s the direction we’re moving in.

Robert Wiblin: So, I’ve also interviewed Paul Christiano who’s working at OpenAI. He was talking about this amplification, what’s the name of this approach that OpenAI has been pioneering lately?

Daniel Ziegler: Yes. The idea with amplification, so going back to human feedback where we have humans evaluating the outcomes of an AI system acting and trying to compare which outcomes are better, we think that’s a step forward, but there’s still a lot of limitations with that because, for a lot of interesting applications, it’s going to be really hard for a human to understand what’s going on well enough to oversee this big, complicated process that might be going on. So humans might, at some point, lose the ability to directly evaluate everything that’s coming out of an AI system.

Daniel Ziegler: The idea with amplification, and we also have a related idea from Geoffrey Irving, another researcher on the Safety Team, around debate is – so basically, imagine we’re trying to build some kind of powerful question and answering system. So you give it some kind of question, say like how good is this transit system design or where should I go on vacation, or something like that. We want to train a system that gives good answers to this kind of thing.

Daniel Ziegler: So let’s take amplification. The idea with amplification is you take some question that you want answered, and one place you can start is to be like, alright, let’s let a human think about that question for 10 minutes and see what they come up with. So you can try to train a system with a bunch of examples of questions, where you give the human some question. You let them think about it for 10 minutes and then they give some answer. You train a system to try to imitate humans on that.

Daniel Ziegler: To be clear, it’s out of reach of current ML systems, but in principle that is a well-defined ML problem. That wouldn’t be that exciting, because if we managed to do that, we’d only have built a system which can imitate a small amount of human reasoning, but not do anything more capable than that.

Daniel Ziegler: So then the idea is, alright, why don’t we, as this person is thinking, why don’t we let them ask some sub-questions. So they get some type of a question like, how good is this transit system design. Then they can break it up into a couple of smaller questions like how much is it going to cost to build, and what’s the economic benefit going to be. Then those sub-questions get to be answered by another human that also gets to think for 10 minutes. That human also gets asked more sub-questions, and so you get this giant tree of little humans, or human-like things thinking for just 10 minutes apiece, but potentially you can actually do a bunch of really interesting reasoning in this gigantic tree. The idea, of course, is that we’re not actually going to have this giant tree of humans doing this really weird computation. We’re actually going to train ML systems to imitate the individual humans in this tree, and we’re also going to train ML systems to try to predict the output of the entire process. So it goes straight from a question to an answer that potentially involved a whole bunch of reasoning.

Daniel Ziegler: The hope is that – I guess the idea behind this kind of system is that we can try to figure out what humans would have said to a particular question if they had been able to think for thousands of years and spawn a bunch of copies themselves and have this big, gigantic deliberation process and then finally come up with an answer. Since we can’t actually do that, we’re going to try to make an ML system do that for us.

Robert Wiblin: What kind of safety work, is there a Google Brain? I guess it’s not the key focus there as much, but…

Catherine Olsson: Well, so, yes. Again, this comes back to what makes AI Safety, AI safety. If you view it as a community of people who refer to their work with that term, there’s really not much happening at Google Brain. If you zoom out and say, well what’s the goal, that one potential way to view the goal of AI safety is to expand the palette of types of outcomes that we can get from AI. If all we can do is build systems that maximize a single number and do so in ways that we can neither inspect nor modify nor have any assurances about, that’s an extremely limited palette, and only quite dismal outcomes will come from that. We’d like to expand that palette to be able to specify things that are harder to specify. We’d like to have more assurances that it’s going to go well, or more ways to inspect that, more ways to modify that, etc.

Catherine Olsson: Chris Olah, who actually just announced that he’s moving to OpenAI, has been doing a lot of fantastic interpretability work. There are other groups within Google, for example the PAIR Group, People + AI Research, that are also doing interpretability work, so that goal is to be able to inspect and understand what systems are doing and, if it appears from your inspection that it’s going to do something that you would like it not to, that’s sort of one avenue for intervening. As I mentioned, robustness and adversarial examples is definitely a focus. There’s a broader security and privacy umbrella, which broadly is looking into how we can make sure that systems classifiers or others are not revealing data about users that we don’t want revealed, or manipulable by adversaries in ways we don’t want them to be. That’s more assurances that these systems are behaving the way that you’d like them to. There’s a lot of fairness work that Google is focusing on. Fairness is an interesting one, where I think it’s not obvious to many people that a system that’s producing unfair outcomes for people is not aligned with our values.

Catherine Olsson: So there is extremely concrete and practical evidence that we already have systems that are not aligned with human values, and if we just crank up the power, deploy them in more contexts, give them more decision-making power, more autonomy, that’s not going away. There are many different angles on this. Another is the work on learning from human feedback. Some of the music and art generation teams are looking into how to incorporate human preferences for what sort of music or art they’d like to generate. That doesn’t appear on the surface to be “safety,” but, again, I sort of view it under the lens of expanding the palette of types of outcomes that we can use AI to achieve as opposed to just “Hammer on this number. Whoops, it didn’t correspond to what we wanted. Oh no.”

Robert Wiblin: So what is the organizational structure that ML groups have? Like what are the different roles that your fill and that other people fill?

Catherine Olsson: So it’s different at different groups. If you’re applying to a specific place, always make sure to ask them, what exactly is this role going to do? Very broadly speaking to try and generalize it across groups, a research scientist is someone who has more experience leading research projects of their own or selecting their own research questions, and is directing that sort of program. A research engineer has more focus on implementation in some sense, so that could be scaling things up or it could be quick, memory-efficient implementations, whatever that might be. Different people come in with different skill sets, so which title you end up in is often just an approximation to what your strengths are when entering.

Catherine Olsson: In my role, there’s no restriction of what I can or can’t do. I’m going to do better at projects that leverage the skills that I have, which are not currently those of picking independently a research direction and leading it on my own, although I am working on those, because I think those would better unlock the set of projects I could work on. Many organizations have more clear-cut structure there where research engineers might get assigned to a particular project that need whatever skills they come in with, but I think it’s often just more useful to look at what skills a given person has as opposed to what title they end up with.

Daniel Ziegler: Yes. At OpenAI, certainly on the Safety Team, it’s a pretty fluid distinction as well. I think that, given the state of ML right now and the fact that it’s such an empirical field, maybe 75% of what an ML researcher will do is exactly the same kinds of things that an ML engineer is going to do, like implementing new machine-learning models and running a bunch of experiments with them, tuning them, fixing bugs, and seeing what works well. Those are all things that I do basically on a day-to-day basis. Then what a research scientist is going to do on top of that is be really up to date with the literature and come up with a bunch of new ideas for what direction to move in.

Daniel Ziegler: Right now, basically on the Safety Team, for the most part, on a high level, I am implementing ideas that some of the research scientists have come up with, but when it comes to the smaller decisions like, how can we try to make this ML algorithm work better, what’s the next experiment to run, sort of day-to-day decisions like that, I am definitely making those decisions myself a lot of the time and could fluidly move into setting more of the high-level vision as well if I wanted to.

Catherine Olsson: For myself, I’ve been doing a mix this past year of working on my own project that I came up with, with help from my manager and other folks contributing to that project too, especially in the last stages after the prototypes had started working, and also jumping in on other people’s projects. I have one skill that I’ve tried to develop, which is to be able to leap into something where I don’t have very much context and spin up quickly and be able to contribute critically so that if colleagues of mine are up against a deadline and need some help in that final mile, being able to jump in and be useful, even though I haven’t had months and months of context.

Robert Wiblin: So I’m mostly familiar with biology labs. How much does it look – like a typical lab would have one PI or one lab leader, and then a bunch PhD students and a bunch of other people who are doing research under them in that kind of hierarchy. Is that kind of similar, or do you have a research scientist who is always overseeing what the engineers are doing?

Catherine Olsson: I don’t think an academic model is a particularly good fit, largely because there are many more research scientists. It’s not quite so much like a PI and grad student type model, and there are research scientists and engineers of all levels who will collaborate in very organic groups with mixes of skill sets. The person with the title of Research Scientist also has a manager, who is also, probably, a research scientist, who they will then go to for advice. I don’t think it’s quite as strict, like once you’re a PI and now you lead your lab and tell your grad students what to do. It’s much more fluid. I think that’s actually a strong benefit. There isn’t in industry labs as much pressure to be the PI of the such-and-such lab. You do have to demonstrate that you are good at leadership and leading independent projects, etc., but there is a lot more flexibility to collaborate in groups of whatever composition that particular project is well suited for.

Daniel Ziegler: I think it varies a whole bunch. I think a lot of the projects that the Safety Team’s worked on have been like one researcher or maybe one researcher and a half working on something that’s their idea but they’re trying to do, and then now with starting to use more natural language, we’re starting to have a slightly bigger team there, so there’s four-ish people working on that, which is new for the Safety Team. There are also a bunch of really big teams at OpenAI, like DotA and robotics, that are just big engineering efforts with lots of people working on them. I think it sort of spans the spectrum.

Robert Wiblin: Yes, so it sounds like the roles are a lot more fluid, perhaps, than what I was imagining that people kind of shift between doing different tasks just based on where they can contribute the most to the project as it is.

Catherine Olsson: Right. Exactly. It’s just very dependent on what a particular project needs, what skills you have, what skills are within reach for you to develop next. It’s a much more organic process, I think.

Robert Wiblin: Yes. I guess I’ve heard people debating about how valuable is it to get someone hired at OpenAI as a research scientist versus a research engineer. It sounds like that might almost just be a confused question, because it’s going to depend so much on the person. In fact, there isn’t a clear barrier.

Catherine Olsson: There’s no clear barrier. Again, it just comes back to skills. I think that question, how valuable is it for a person to be in role A versus role B, that’s what skill would be implied by the person getting hired as A or B. I think folks who are able to come up with new research agendas that are well targeted at problems that need solving are, of course, incredibly valuable, and so are folks who can jump on existing agendas and push them forward. They’re valuable in different ways. I think as the set of questions under the safety umbrella evolves and matures, then different skills will be needed in different parts of the problem. That’s something that I was pointing at before, that right now there are corners of the space that you don’t even need any machine learning experience at all to contribute to. There are other parts of the space that would be difficult to get a hand hold on, unless you’ve already demonstrated that you can productively do novel research in unexplored domains. These are very different skills that are needed.

Daniel Ziegler: Yes, and I think that, right now, on the OpenAI Safety Team, we’re at a point where we’re trying to scale things up and we can absolutely use just more engineering effort. I think – so when I joined, I was the first research engineer. It seemed pretty clear that the work I was doing was being very directly taken off the hands of some of the people with “Research Scientist” in their title, and I was able to let them, like let Paul Christiano and let Geoffrey Irving think more on a conceptual level about how they wanted to build their alignment schemes. So both from the perspective of driving experimentation faster with more concrete prototypes of the schemes and from the perspective of giving people more time to think on a more abstract level, I think it was really valuable to have me there as a research engineer.

Robert Wiblin: Yes. Do you think there’s a difference in replaceability between, you know, I mean there’s people who have whatever schools come with having finished a PhD and those who don’t? If they hadn’t been able to hire you, what would have been the next best alternative to them? Was it someone who’s significantly worse or just not making a hire at all, or is it someone who’s just marginally less as effective? Perhaps more generally, like some people it’s easier to find a second-best candidate than others?

Daniel Ziegler: Yes. I do think it’s going to vary a lot. Right now on the Safety Team, both seem very not replaceable. I think if I hadn’t been hired, that would have just been one fewer research engineer for at least the next half year or so. Although, as we continue to grow and reach a bigger hiring pool, that might change to some extent. I think the same thing is true, and probably even more so, for people that are able to contribute more of their own agenda. I think that’s something that we can always use more of and there will be room for that for a long time.

Catherine Olsson: I’d additionally like to emphasize management skill. That’s something that I think hasn’t come up here yet, but a lot of research teams need folks who are both technically competent and good managers. So it’s not just like, can you write tensorflow code versus come up with ideas, it’s also can you build out a team or manage a project. Those are also incredibly valuable skills. If you’ve got some of one and some of the other, that’ll also go a long way.

Robert Wiblin: That’s kind of a potentially unique combination that’s very hard to hire for otherwise.

Catherine Olsson: And additional pieces like people who are excited about thinking about the strategic vision of what an organization or a team should do, or where is humanity going with all of this, if you’re willing to engage both at the level of debugging tensorflow and at the level of maybe more policy-like questions, that’s another difficult to find combination.

Robert Wiblin: Do you think either of you are learning more on this job than you would have if you’d done the PhD, or continue the PhD?

Catherine Olsson: Absolutely.

Daniel Ziegler: Yes I think so.

Robert Wiblin: It seems that way.

Catherine Olsson: I think the best way to figure out what’s going on is just to dive in. In fact, I’m directly referencing a post by Nate Soares, called Dive In, which I love and recommend, that if you have an extremely concrete plan of how you’re going to contribute that has actionable and trackable steps, you’re going to start getting data from the world about your plan a lot sooner than if you have some unreachable or nebulous plan. I would encourage anyone who’s interested in this sort of thing to look for the smallest step that you can take that brings you just a little closer. If you’re currently a software engineer and you can take a statistics class and maybe do some data science in your current role, by all means do that. Take just one step closer to something in the space of machine learning.

Catherine Olsson: If you can just do software engineering at an organization that does ML, now you’ve just, if you take that role, you’ve just got your face in the data in a much more concrete and tangible way. I think, particularly folks who are coming at this topic from an EA angle, maybe you’ve read Superintelligence, whatever your first intro was, those abstractions or motivating examples are quite far removed from the actual work that’s being done and the types of systems that are being deployed today. I think starting to bridge that conceptual gap is one of the best things that you can do for yourself if you’re interested in starting to contribute.

Daniel Ziegler: Yes, and I would say, try just diving in all the way if you can. Like I said, when I was preparing for the OpenAI interviews, I went straight to just implementing a bunch of deep reinforcement learning algorithms as very nearly my first serious project in machine learning, and obviously there were things along the way where I had to shore up on some of the machine learning basics and some probability and statistics and linear algebra and so forth, but by doing it in sort of a depth-first manner, like where I just went right for it and then saw as I went what I needed to do, I was able to be a lot more efficient about it and also just actually practice the thing that I wanted to be doing.

Catherine Olsson: Yes, I definitely second that. Anything that you can do that’s hands on and tractable is going to get you a lot farther. One mistake I see people make is getting very intimidated by these very long reading lists. There are many reading lists out there of how to get started in AI safety, read these 12 books, and they’re like, oh my God. People fall into the trap of being like, oh I’m going to learn reinforcement learning. I’d say, swap that out. Rather than learn X, try to learn to do X. I’m going to learn how to implement DQN on Atari. Great, now you can tell if you’re on track for that, whereas if you’re trying to “learn RL,” you have no way to know, have I learned it yet or not? Whereas if there’s something you’re trying to do, then you can tell, can I do that thing yet or not.

Robert Wiblin: So another question I’ve heard people discuss is, in as much as you’re trying to get an organization to take safety considerations more seriously, it might be the case that research scientists have more influence over the ideas of the culture, their priorities. Do you think that’s true, or is it just kind of anyone there who is contributing who has good ideas, they can get taken up.

Daniel Ziegler: I think, for the most part, it is the latter. I think that – I feel like I’m in a position where if I have opinions on things or ideas for where the work should go, I will at least get listened to. I think it wouldn’t be that different if I was a research scientist. The more people in an organization that are thinking about safety and really have the long term in mind, the better. I think that even if they don’t have direct decision-making influence, just by being there and talking to people, you can make sure that an organization is moving in the right direction.

Catherine Olsson: I think, sort of to break down what does it mean to “take safety seriously”. That has two quite different pieces right now. One is, as a research-producing organization or institution, to focus on research that will do what? It will improve robustness or shape our ability to inspect or provide feedback, whatever it is, so there’s that one question of “take seriously” as in “prioritize research with a certain flavor.” Then there’s another of being cognizant of the effects on humanity of anything that you actually deploy. Those are very, very different. What’s being researched and what’s being deployed and “taking seriously” looks very different in those cases. Some of those are both extremely valuable and I think from the research standpoint, in either of these roles you have some chance to choose what you work on and you can choose to put the firepower of your output behind projects that you think are most valuable in any role that you’re in.

Catherine Olsson: On the question of what’s actually getting deployed, at a place like Google, even if you’re just a Google software engineer on Cloud or something, you can be feeding into the organization’s procedures around choosing what’s good to build and what’s good to deploy. That’s actually accessible to people who are totally outside the scope of research – at the edge of what’s getting deployed to real users. I think that’s also valuable, though it’s quite different from the research side.

Robert Wiblin: Yes. I relay the conversation that I’ve heard is, is there much value in being someone who works some kind of capabilities within these organizations, where it’s not clear that you’re having either of these effects directly that you’re not part of a safety focused team, nor do you seem to really have that much influence over deployment per se, but still you’re part of this broader, important organization that could be influential within the development of machine learning, and you have a particular view about how safety should be given a lot of importance, relative to just speeding things up.

Robert Wiblin: Do you have any perspective having been in these organizations of how useful that is?

Daniel Ziegler: Yes, I kind of want to repeat what I said earlier about just the more people in an organization thinking about these issues the better. One great thing about OpenAI is that throughout the company, I think the safety team is really well respected. It’s a really strong team and it has a bunch of strong people on it. It probably helps that we’re doing legitimate-seeming ML research, but I think having people outside of this sort of nominal safety team also caring about that, can help that situation come about, like if you’re at a place where people working specifically on safety related things don’t have as much respect as other parts of the organization, that could help. That said, I think it does seem significantly more valuable to actually be working on stuff directly.

Catherine Olsson: I would say that if your goal is to have some positive impact on the world, you need some plan for what exactly that’s going to look like. Simply being part of an organization and abstractly caring about something is not an impact plan. There are many ways that could go that could have a positive impact, like get involved in projects that don’t seem directly relevant to the stuff you care most about as a way to get related experience if you think there’s better mentorship on that project. That definitely works. You could work your way up the ladder of influence in the organization, and then hope to shape how decisionmaking in this organization happens, in general, at the high level and sort of steer an organization towards what you view as more safety promoting or positive outcome promoting, decision-making structures in the organization.

Catherine Olsson: Now I’d hope that if you want to do that, that you actually know something about which decision-making structures in an organization go better or worse. This is a case where simply standing nearby and caring is not going to have any particular impact one way or the other. But if you become an expert in organizational decision making and what causes organizations to make safe versus unsafe decisions and then put yourself in a position of power, that sounds fantastic. I would love every organization to have people with actual expertise about organizational decisionmaking shaping those decisions. I think simply being aware that there is a problem is not even remotely the same thing as having concrete and particular skills that will cause outcomes to be better.

Robert Wiblin: Okay, so dealing with a slightly different topic now, what does your day look like? How much time do you spend, I guess, in the office? How is that roughly divided between different kinds of tasks?

Daniel Ziegler: I tend to spend, because I care about what I’m doing, I tend to spend maybe ten hours a day in the office, although I’m just currently taking weekends off. But, yeah, so, I mostly just work on coding up new stuff in Python, in TensorFlow, new machine learning code, running that code in a whole bunch of experiments for different tasks or different hyperparameter settings to see how well it’s working and then based on the results of those experiments, going back and tweaking something to make it better, fixing bugs, and those kinds of things. Some of the time, I’ll be working on something that’s completely new.

Daniel Ziegler: We’ll be solving a task we haven’t approached yet or adding some new twist to the machine learning problem, or we’ll just have some existing benchmark that we’re trying to do better at, and it’s just a matter of making things run faster or improving the training process to make it more and more effectively, so it’s both being able to do new things and doing better at existing tasks. It’s a lot of back and forth. Sometimes, I try to have two things I’m working on at the same time so I can code on one and fire off the experiments. While those experiments are running, go back to the other thing. It’s a little bit multitask-y, and something I’m still figuring out how to deal with, but, yeah, that’s the bulk of what I do.

Catherine Olsson: Similarly, I spend the bulk of my uninterrupted times, I at least strive to spend coding because that’s the most valuable time for code. Any given project looks different at different phases of its life cycle. I also try to have two projects, one that might be a little more just put in the hours, and you know what you have to build, and you just have to build it, and another that might be more experimental where you’re throwing together prototypes or reading things that are related, or talking to people about those ideas.

Catherine Olsson: I also spend maybe 20% of my time on what I think of as more in the vein of policy or outreach about what our team is doing and how that’s important both internally and externally. So, some of that within Google working on groups that are looking at how do we implement the security and privacy principles? What sort of trainings might we need? Or, going to external workshops that are looking for the perspective of folks who know how adversarial examples or ML security fits into that picture.

Robert Wiblin: Have either of you considered working at DeepMind or other organizations and how would one decide which organization to go to if you had a choice?

Catherine Olsson: Personally, I don’t want to move to London right now.

Daniel Ziegler: Same.

Catherine Olsson: So, I’m not inclined to go to DeepMind, but I hear great things about them if you’d like to go to London.

Daniel Ziegler: Yeah, I know a couple of people on the safety team at DeepMind, at the ML security team at DeepMind. I think they’re definitely doing valuable work as well, but I think you should really just look at some of the work that these teams have been doing and see what seems the most valuable to you and also the most exciting for you to work on yourself, then see if you can handle moving to wherever the thing is.

Catherine Olsson: I also want to emphasize again that just because … Not everything that’s good to do is labeled AI safety. If you restrict yourself just to things that are called a safety team, you’re going to miss a lot of important problems, a lot of good mentorship that’s not under that umbrella. Of course, it’s easy to just do … I think there’s also the flip side. It’s easy to do something that just rhymes with the problems that you care about, but isn’t actually going to contribute. That can be an okay way to get some related experience. Right, so, one thing to highlight is many research orgs do some sort of residency or fellowship program.

Catherine Olsson: The Google AI residency is one of the more well known ones, but there are many year long programs like this that’ll spin people up on research. Don’t restrict yourselves just to the ones that are in the orgs that immediately come to mind, because there’s plenty of these sorts of fellowship or training programs. Similarly, any given research question that you find interesting, I’m sure there are many groups that are tackling it from different angles. So, if you have some question in mind, you can go from there rather than just trying to key off, is it called a safety team or not?

Robert Wiblin: Yeah, are there any specific projects that you think are underrated or people aren’t sufficiently aware of?

Daniel Ziegler: One organization I want to plug is Ought run by Andreas Stuhlmüller which is very related to some of the stuff we’re working on on the safety team at OpenAI, but it’s trying to do human experiments around amplification in particular. It’s trying to do experiments to see how does it work when you try to break up a question into smaller sub questions and only have any individual person think for a small amount of time, but have the thing as a whole involve a whole bunch of thinking. I think they’ve been looking to hire research engineers for a while. That might be a place you haven’t heard of. Yeah, the output of their work will be super useful for informing how we want to continue doing research on the safety team.

Catherine Olsson: Yeah, I think one area that people often overlook is verification of neural nets. That’s happening in a lot of different places, but, yeah, I think verification of neural nets is under appreciated. There’s been some early progress. It doesn’t quite work at scale yet, but it could. That would be a fantastic tool to have in our tool kit.

Robert Wiblin: Where do you think you might go next after your current position or project?

Catherine Olsson: Good question. The place that I’m currently trying to go with my career is a combination of having good technical chops like implementation skills, some sense, some nose for research direction, and also management skills. I’d like to be able to empower teams to work better. I’m doing as much of that as I can in my current role, but I definitely would like to move towards a management role in the future.

Daniel Ziegler: I’d really like to move in a direction where I’m doing more research-y work and and coming up with my own ideas, my own agenda. I think it’s both the case that those ideas would be really useful on their own right, but also just having that kind of big picture vision would help me make better low level decisions when I’m doing engineering type work as well. I’ve noticed that it’s really important to understand what the goal actually is and what we’re trying to aim for when we’re trying to develop a system. I can have an alright idea of that and make some calls of my own, but the better my understanding is there, the more effectively I can work on my own, and the more effectively I can make the right decisions about what’s the right tweak we can try next, what’s the number I should actually be trying to optimize, and so forth?

Robert Wiblin: What are the biggest disagreements between people who are interested in alignment in machine learning? Did you have any view on it? Are there any controversial questions that people have heated conversations about?

Catherine Olsson: The primary disagreement that comes to mind is not so much a disagreement you’d have a heated conversation about as just a difference in vision is what should we build? If you start from the classic AI safety thought experiments, they basically boil down to if we were to build some sort of long range goal planning agent with high autonomy that can reason consequentially and is optimizing for a single objective, that will go badly. Yeah, no one’s about to build one of these. What should we build instead?

Catherine Olsson: I think that negative space is actually a much richer space than people realize. Sure, there’s one thing that would be really bad to build, but what else? What, of the many other things, could we or should we build? I think that leads to a lot of difference in what you see in different agendas. A system like [Paul’s 01:09:36] amplification project is not necessarily going to go around the world or go around your house and clean your dishes. It’s just not that kind of system. Should we have agents that wash dishes? What should they be like? Should we have agents that make important decisions in major world powers? What should those be like?

Catherine Olsson: Is that even a good idea at all? Should we just make sure we never build those? I think these differences in vision for the future are actually pretty substantial in people’s motivations in terms of what they work on, what they envision building. I think they often go unsaid that we can sweep all this under the rug of, oh, well, we all just want to build something such that it goes well for everyone. The negative space of “go poorly” is actually huge.

Daniel Ziegler: Yeah, I would totally agree with that. On the safety team, we’re also definitely not trying to build a monolithic super-intelligence which is going to rule the world. We are also definitely hoping to have a much more gradual transition where we build systems which are usable to enhance our capabilities and give us useful advice and slowly take us to a world where more and more things are being optimized with AI systems somewhere in the loop, but do that in a safe manner and without suddenly transforming the world.

Daniel Ziegler: In general, there’s not that much of a reason to deploy things that are actually agents and acting on their own right in the world. Although, I think you get a lot of safety problems even if you just have systems that are sort of giving humans advice. If humans don’t understand why decisions are being made or why a certain thing was being recommended, they could still end up causing a lot of damage even if the AI system isn’t actually acting on its own.

Catherine Olsson: I’ll also point out in a similar vein that I think there are some non-disagreements that appear to be disagreements just because of differences in framing. So, from the community that would call itself the AI safety community, there’s a focus AGI as an example of an extremely transformative technology, which people think would go badly by default. From other communities within ML, there’s more of a focus on intelligence amplification. “Let’s broaden the scope away from these terminator-like, autonomous, powerful systems, and focus on other ways that automated decision making can make people’s lives better.”

Catherine Olsson: These are extraordinarily compatible views. They’re almost the same view, that excessive focus on powerful, autonomous systems is going to go badly. Right, that’s exactly what both communities believe, but there are differences in emphasis and differences in language that obscure fundamental commonalities in vision, I think, of what would be good to do, right? Where ought we go as humanity? What would it look like for us to thrive?

Robert Wiblin: Has learning more and more made you more or less worried about how the future is going to go, specifically about ML, but maybe more broadly?

Catherine Olsson: Initially, when I first moved into the field, definitely more. I came from a place of … “Some people seem worried. I wonder what’s that about,” to, “Wow, there’s so many ways this could go badly, different scales and types and scopes and trajectories of badly, but there’s many, many different ways for it to go badly.” I’d actually say that’s still the core of my motivation. People talk about existential risk, but there’s also just ordinary catastrophes are entirely possible here. Geopolitical strife over perceived strong systems, even if they’re not, could be quite bad, all the way down to just ordinary surveillance is pretty goddamn bad.

Catherine Olsson: Many, many different ways that things could go badly if sufficient care isn’t taken, that gets me worried across that spectrum. Now, there’s of course reason to then say, well, okay, these are of different scopes or scales. I think on that level, my thoughts are still evolving in terms of what trajectory is humanity likely to be on and how can we steer away from the worst blunders? But, I think the overall spectrum is becoming more and more clear to me that making things solidly good is actually quite hard, and there’s many different kinds of badly we could end up in.

Daniel Ziegler: Yeah, I think I followed a somewhat similar trajectory where in the beginning AI safety seemed like this kind of … When not a lot of people were thinking about it seemed like this fringe idea that seemed almost maybe a little bit crazy, but just learning a lot more arguments and a lot more concrete ways that things could go badly definitely made me more worried. I think one thing that’s made me a lot more optimistic is that I am pretty excited about a good chunk of the work that is being done, including on the OpenAI safety team. I think we do have some approaches that actually do have a chance of making the situation a lot better. That’s made me a lot more optimistic.

Robert Wiblin: Yeah, do you want to talk anymore about the kind of AI policy and strategy side of things? We’ve had conversations with Miles Brundage and Allan Dafoe. Could you imagine going into those kinds of questions at some point in the future? Or, are you very enthusiastic for other people to do it?

Daniel Ziegler: Definitely enthusiastic for other people to do it. Including at OpenAI, actually, Miles Brundage just joined OpenAI as well to work on all those kinds of questions there. Yeah, it’s something that I want to think about for myself and get a better understanding of. I don’t think it’s something I’m going to make my primary focus, but I do think that technical directions absolutely need to be informed by broader policy and governance thinking. It’s also the case that they sort of trade off against each other to some extent. The more we can enable good global coordination and have strong international institutions for safely developing and employing AI, the easier that makes the technical problem. We’ll get more time to solve safety problems and won’t have to think about being in some kind of race dynamic. Yeah, just in general, the easier it’ll make the situation.

Catherine Olsson: I’m really glad that folks like Miles and Allan Dafoe are taking a very sophisticated approach to these questions. Any system that actually gets deployed in the real world is going to get deployed in a sociocultural context with all of its own complexities and nuance. I think it’s really important for those of us who work on the technical side of things to remember that context is extremely complicated. My training is not in international geopolitics, yet the work that I do has those implications.

Catherine Olsson: I think it’s important to remember how important the whole rest of the world with all of its sociocultural complexity is in terms of the context for the technical work that’s getting done. Whether I might go work on that sort of stuff in the future or continue down the path of doing technical research, I think it’s important to me to stay in touch with “what are researchers on the political or sociological or ethical side of things saying?” about the real world impact of these technologies.

Robert Wiblin: What would you say to people who are skeptical about the tractability of working on AI safety at this point?

Daniel Ziegler: I think it’s definitely correct to be worried about tractability, but I think it’s not a question that can be answered completely in the abstract. You really want to look at some of the ideas that are out there for trying to make the future of AI more beneficial and think about how useful do those seem. I think that your opinions can definitely vary on that, but I do think people have a lot of ideas, so it’s worth looking at those. I also want to say I think the common argument against tractability is something like we don’t know what really powerful ML are going to look like, so it’s really hard to work on them now.

Daniel Ziegler: But, first of all, I’d say it’s not that crazy to suppose that something like the current set of deep learning techniques could be extended to make some really, really powerful systems that have a lot of impact. We keep seeing more and more problems just getting solved by just our current techniques, which are still really stupid in a lot of ways, but, yet, they seem to be enough to do a lot of things. It’s also the case that certainly the kinds of things we work on in the safety team are pretty general. It doesn’t actually matter that it’s deep neural networks that are in our systems.

Daniel Ziegler: Any kind of black box system that has some kind of loss function and a bunch of training data, we could just plug into our schemes. If there’s a bunch of advances and people pretty, pretty radically change the way they do machine learning, as long as it still is machine learning in some sense, I think what we’re doing today will still mostly apply. Of course, some of the specific engineering details that we’re trying out now aren’t going to be relevant anymore, but I think that’s okay.

Daniel Ziegler: I think it’s really important to do really concrete experiments with prototype ideas when we can just to get some exposure to real data and have a real feedback loop that shows us which parts of these schemes are important, which things don’t seem to matter that much, at least at the current scales. So, there are lots of useful things, empirical things we can try. But, that doesn’t mean that everything’s going to get thrown out the window when a bunch of advancements happen.

Catherine Olsson: One thing I’ll emphasize on the topic of tractability, which I mentioned earlier is the idea of a scientific paradigm as a happy marriage between a set of questions and a set of tools that produce demonstrable progress on those questions. I think in the context of a happy and functioning paradigm, there’s plenty of tractable work to do for people with those skills. When it comes to how do we go to a pre-paradigm question and bring about the first paradigm in that question, I don’t think humanity knows how to do that reliably. So, that’s pretty untractable.

Catherine Olsson: Some people seem to be able to do it, but I think keeping that sort of distinction separate is important. Also, not everything that currently is operating as a paradigm is necessarily pointed right at the crux of the problem. Every abstraction that humanity has used has been in some ways incomplete, and yet useful. I think striking that balance between what are the useful abstractions that we can make technical or theoretical progress on, that yet are close enough to the types of real things that we want to see or cause in the world is maybe just the whole problem of science.

Robert Wiblin: Okay, so, let’s move on to the more specialized question for people who are really considering taking action and potentially copying your example or doing something similar. I guess a lot of the lessons that we might be looking at here have kind of already come up somewhat indirectly by looking at your experiences, but let’s make them super concrete and flesh them out even more. What do you think is required to be a good ML engineer? Are there any concrete signs that people can look at to say, yes, I could do this, or, no, I should try to do something else?

Catherine Olsson: Could I actually just not answer your question, to say the first thing that came to mind based on what you said there? One thing folks should keep in mind when looking at my experience, but I think many successful people’s experience is that it was a huge component of luck. Don’t discount that, right? If I had decided to quit grad school in a different year, OpenAI would not have been hiring rapidly.

Catherine Olsson: But, it just so happens that the moment when I decided to change tracks was the moment where OpenAI needed to hire people with my skill set. That kind of opportunity doesn’t come up all the time, and my life would have looked different if it hadn’t. So, if you’re finding that you’re struggling, it might not be that you lack the skills. It might be that window of opportunity hasn’t come around for you. So, don’t get discouraged, particularly if you’re comparing yourself to folks who you perceive as successful. Often, there was a large component of happenstance.

Catherine Olsson: Of course, you can take steps to have those opportunities come to you more often. You can talk to people who are important in the field or write blog posts that catch people’s eye, or go viral on Twitter, or otherwise bring some spotlight of attention to yourself, but I don’t want to downplay the importance of just the pure happenstance of people’s trajectory who … What companies happen to be hiring for what, or what past experiences they might have had.

Daniel Ziegler: Yeah, I definitely agree with that.

Robert Wiblin: So, setting that aside, setting aside that caveat-

Catherine Olsson: I just want that to be somewhere in there.

Robert Wiblin: No, that makes some sense. You can’t only look at the … Well, we have to look at other people who took your strategy. What’s the full distribution around them, just sampling from the top? You don’t want to select based on the Y axis, but hopefully there’s still some wisdom we can get. Perhaps you can look more broadly at other people you know in the field and what choices they’ve made and how things have gone rather than just looking at your experiences. Are there any indications of things that are a good sign about your prospects?

Daniel Ziegler: Yeah, I think there is a couple of sorts of skills that you are going to need to have. You do need to be a pretty good software engineer. You do need to have some understanding of machine learning and statistics at least by the time you actually are going to do the job. But, honestly, I feel like the best thing to do is try it. You can spend a couple of days just trying to implement, trying to replicate some machine learning paper. You’ll notice pretty quickly whether it’s really painful or really, really difficult for you, or whether it’s something that you can imagine doing. It’s always frustrating, even for people who are very good at it. Working with ML, it can just be a pretty big pain. Part of that is having the right degree of frustration tolerance, but all these things are things you can find out by giving it a go.

Catherine Olsson: The types of software skills that you need to work on machine learning are similar or overlapping with but not identical to the skills that make a software engineer who’s writing production code good at what they do. You need iteration speed. Tom Brown, who’s a coworker of mine who’s one of the more successful research engineers that I know was previously a startup engineer building web startups where iteration speed is the primary driver. If you can iterate quickly, even if it seems to you to be in an unrelated technical or coding domain, that’s actually quite a good sign. Also, the kind of good code that you need to write is not necessarily exhaustively tested so much as clear and quick to read and verify that it’s doing what you want. These are very different definitions of good. I think machine learning code, at least research code, is a lot more about being nimble than it is about being exhaustive.

Daniel Ziegler: Yeah, it’s really hard to test machine learning code well. I think basically debugging it can be quite difficult, and the best you can do is hope to make it really easy to read so you can verify that it’s actually doing the right thing, and then collect a bunch of metrics on how it’s performing to try to get some signal on whether things roughly look the way that they should. But, you’re not going to be able to write tests or verify it the way you could other kinds of software.

Robert Wiblin: What fraction of people working as software developers more generally do you think could make transitions into similar roles to the ones that you’re doing?

Catherine Olsson: I’m going to go out on a limb and say most, to be honest. I think one barrier that many software engineers have to transitioning to a ML software role is the time, freedom, flexibility, et cetera, to do that retraining. It does take a while. That’s part of why I suggest that if folks can, in their current role or their current organization, start to gain machine learning skills, to do it that way if they can’t take the time off.

Catherine Olsson: Maybe they don’t have the runway, or they have other obligations, that there are ways that you can start seeing how machine learning fits for you if you’re already in a software role, because many organizations want to be using more machine learning. If you go to your manager, you’re like, “Hey, I want to go retrain into machine learning so that I can apply it to our such and such pipeline,” I bet many managers will be like, “Oh, yeah, great, great. Go, take these trainings. Go learn this stuff.”

Catherine Olsson: I think even if you find it hard to imagine taking three months off and studying machine learning full time for those months, you might be able to find a way to work that into your current role. I think looking for those opportunities could help more people find that sort of path. But, if your current situation doesn’t give you that flexibility, again, I have a “don’t blame yourself” thing here. If you don’t have the luck, or if you don’t have the flexibility, transitioning into machine learning does take some of that flexibility that not everyone has the privilege to have.

Robert Wiblin: I think a common thing among potential software engineers or software developers listening to this will just be that they kind of feel under confident, or I see this a lot, of people who are too scared to make this transition. I guess for that reason, I’m just looking for concrete things that potentially a substantial fraction of listeners might have that will be, yes, this is really a sign that you could do this, and there’s a good chance of success. Is there anything, I don’t know, even academic results that most people will have had at some point that are an indication of capability?

Daniel Ziegler: It’s hard to say. I want to say that I was actually … I actually felt quite under confident basically until I actually got the job offers from MIRI and OpenAI. I really was not sure that I was going to be able to make this transition well and land in a position that I thought was really valuable. I definitely think I both got pretty lucky there and also, yeah, I absolutely wasn’t sure. I was in a pretty, pretty privileged position to be able to spend a couple months trying it. I do think it was a good sign that I did have a strong computer science background in general. I had some research experience, even in a different field. That had gone pretty well. Those certainly are good signs, but they’re not entirely necessary, either.

Catherine Olsson: I’d just like to emphasize that machine learning is not that hard. It sounds really intimidating. It’s just not that hard. I’ll also make a plug for the [fast.ai 01:27:20] course. There’s a lot of [MOOCs 01:27:23] or online courses out there, but that one seems to be particularly well tuned for getting people hands on experience, quickly hitting the ground running. For f