Introduction/Summary



2015 is the year when AI alignment and AI safety took off as causes taken seriously by the academic and industry worlds of AI research and the media. This has largely been attributed to influential world leaders in science and technology like Bill Gates, Stephen Hawking and Elon Musk. They all cited Nick Bostrom’s book Superintelligence as having changed brought the serious issue of AI alignment to their attention. Published in 2014, Superintelligence gets the credit it deserves for having put AI alignment on the map like never before, along with significant help from the Future of Life Institute. However, the AI alignment, rationality and effective altruism (EA) communities at large aren’t credited enough for what they did to advance serious attention to the cause. My impression having talked to many people is that the popular history is AI alignment lucked into hitting a tipping point. There’s little written up about the history of AI alignment as a cause developed by the rationality and effective altruism communities, and associated organizations. This is important because if we don't keep track of how much of AI alignment's rise to prominence is due to deliberate effort from within the community, we won't know how lessons of what worked can be applied to other fields.This observation has been made by effective altruists wanting to develop other fields, such as welfare biology Having been involved in the rationality and EA communities for years before the publication of Superintelligence, I saw the build-up of AI alignment as a cause first-hand. Between this experience and a few key sources which highlight important points in the history of AI alignment as a cause, I’ve noticed its development can be broken down into multiple stages. In this post I aim to explain how various strategies for growth and development over the course of AI alignment’s history can be generalized to other causes.



A Brief Look Back At AI Alignment



As mentioned above, the Future of Life Institute (FLI) played a crucial role in working with Nick Bostrom and capitalizing on the publication of his book Superintelligence. Much of FLI’s work when it was first founded was focused on organizing a conference on AI alignment a month after the publication of Superintelligence, to which were invited journalists, academic and industry leaders, and public figures such as Elon Musk. The work of relating and communicating AI alignment to the public while trying to ensure fidelity of messaging FLI has done behind the scenes in 2015 and before is illustrated in this 2015 overview on their website.



Max Tegmark, co-founder of FLI, explains the role the Center for Applied Rationality (CFAR) played in FLI’s creation:

CFAR was instrumental in the birth of the Future of Life Institute: 4 of our 5 co-founders are CFAR alumni, and seeing so many talented idealistic people motivated to make the world more rational gave me confidence that we could succeed with our audacious goals."



CFAR in turn owes its existence to Less Wrong and the rationality community. LessWrong started with the Sequences by Eliezer Yudkowsky, which also contains his foundational contributions to the theory underpinning AI alignment. Anna Salamon, co-founder and president of CFAR, describes in her post On the Importance of Less Wrong, the crucial role Less Wrong has played in the existential risk (x-risk) and AI alignment communities:





One feature that is pretty helpful here, is if we somehow maintain a single "conversation", rather than a bunch of people separately having thoughts and sometimes taking inspiration from one another. By "a conversation", I mean a space where people can e.g. reply to one another; rely on shared jargon/shorthand/concepts; build on arguments that have been established in common as probably-valid; point out apparent errors and then have that pointing-out be actually taken into account or else replied-to).



One feature that really helps things be "a conversation" in this way, is if there is a single Schelling set of posts/etc. that people (in the relevant community/conversation) are supposed to read, and can be assumed to have read. Less Wrong used to be a such place; right now there is no such place; it seems to me highly desirable to form a new such place if we can.

Looking at all these historical efforts to build up a community around existential risk reduction and AI alignment as a single long-term project, I’ve noticed multiple stages of community build-up around these fields.



Stages of Developing a Movement or Field

In hindsight, we can look at these historical efforts to build up a community around the ideas of existential risk reduction and AI alignment as a single, long-term project. Viewed this way, it appears there are multiple distinct stages of community development in the history of AI alignment. These stages can be generalized to examples from other social and intellectual movements as well, such as animal advocacy and effective altruism.

Applying This Approach

Each of the three major focus areas of effective altruism:



Global poverty alleviation; primary means: public health & economic development



Existential risk reduction; primary means: AI alignment & global coordination



Effective animal advocacy/welfare; primary means: multi-pronged approach to mitigating animal suffering due to factory farming



benefited from another community focused on similar goals before effective altruism existed. Movements like transhumanism, the rationality community, veganism, animal welfare, and animal rights/liberation focused on EA causes for years before EA existed. Global poverty reduction has been a goal of charity worldwide for thousands of years, with its transformation into modern philanthropy benefiting from decades of research from fields like economics and epidemiology. Since its inception several years ago, EA has internally fostered the growth and development of some causes which to this day outside the movement receive little public interest. Notable examples include the focus areas of wild animal suffering/welfare and life extension. Foci like existential risks in addition to AI alignment; emerging technology R&D; public policy reform; and mass mental health interventions have begun receiving more attention from the EA movement in the last couple years.



As effective altruism has been constructed from the confluence of so many prior social movements, intellectual fields and online communities, as a community we have the collective experience to deliberately repeat this process. EA as a sizable coordinated network has the capacity to devote more resources to grow and develop smaller causes more rapidly than it was before. With the benefit of hindsight we also have more knowledge and ability to steer the intellectual development and project coordination for all kinds of causes. Looking back at the history of AI alignment and existential reduction, its been over a decade to get to its current stage. Other EA causes benefited from decades of public interest to get to their current stages of development. Ideally, smaller causes will be able to employ the above strategies in time to develop even more rapidly than the biggest focus areas in EA at present.



What's Next?

Having recognized these common and effective strategies for community-building, I've begun collaborating with the intellectual communities around various causes on the first stage of field development, Knowledge Production. I've been doing this for the following focus areas in the following respective Facebook groups.



Over the last couple months, I've begun compiling research materials on these subjects to form bodies of research I intend to post on the Effective Altruism Forum when complete. Depending on the focus area, they may be cross-posted on other websites as well. You're invited to join any of these groups to follow or contribute to any of these projects. Also feel free to suggest any other cause which would benefit from such a project. Having compiled a comprehensive spread of quality research for various fields and posted then to the EA Forum, I hope this information will be used to further advance and develop causes within effective altruism.





