Co-authored with Jonah Sinick.

How big is the field of AI, and how big was it in the past?

This question is relevant to several issues in AGI safety strategy. To name just two examples:

AI forecasting . Some people forecast AI progress by looking at how much has been accomplished for each calendar year of research. But as inputs to AI progress, (1) AI funding, (2) quality-adjusted researcher years (QARYs), and (3) computing power are more relevant than calendar years. To use these metrics to predict future AI progress, we need to know how many dollars and QARYs and computing cycles at various times in the past have been required to produce the observed progress in AI thus far.

. Some people forecast AI progress by looking at how much has been accomplished for each calendar year of research. But as inputs to AI progress, (1) AI funding, (2) quality-adjusted researcher years (QARYs), and (3) computing power are more relevant than calendar years. To use these metrics to predict future AI progress, we need to know how many dollars and QARYs and computing cycles at various times in the past have been required to produce the observed progress in AI thus far. Leverage points. If most AI research funding comes from relatively few funders, or if most research is produced by relatively few research groups, then these may represent high-value leverage points through which one might influence the field as a whole, e.g. to be more concerned with the long-term social consequences of AI.

For these reasons and more, MIRI recently investigated the current size and past growth of the AI field. This blog post summarizes our initial findings, which are meant to provide a “quick and dirty” launchpad for future, more thorough research into the topic.

To begin, we tried to quantify the size and past growth of the field using metrics such as

Number of researchers

Number of journals

Publication counts

Number of conferences

Number of organizations

Famous prizes awarded for AI research

Amount of funding

It’s difficult to interpret these figures, and they may be significantly less informative than an object level study of the research would be, but the figures still have some relevance:

For the purpose of investigating growth, one can look at year-to-year percentage growth in the statistics, combining this with other measures of the amount of progress that has occurred in AI, in order to estimate the amount of AI research that will occur in the medium term future.

For the purpose of investigating the current size of the AI field, one can look at the quantitative metrics relative to the corresponding metrics for computer science (CS) , and use these in conjunction with a holistic sense of the current size of the CS field to inform one’s holistic sense of the amount of progress that there’s been in AI.

The data that we were able to collect provide a decent picture of the size of the AI field relative to the size of the CS field, but they are insufficient to support a robust conclusion, and more investigation is warranted. Unless otherwise specified, see the spreadsheet “Current size & past growth of AI field” for the raw data on which this blog post is based.

The size of the AI field

According to a variety of metrics, the amount of AI research being done appears to be about 10% of the amount of computer science (CS) research being done. The metrics used, however, mostly capture research quantity rather than research quality, and thus may be a weak proxy for measuring how many QARYs have been invested. That said, the fact that roughly 10% of CS research prizes are awarded for AI work may indicate that research quality is similar in CS and AI.

We obtained many of the relevant figures from Microsoft Academic Search (MAS). MAS allows one to search under the headings:

Computer science

Artificial intelligence

Natural language and speech

Machine learning and pattern recognition

Computer vision

One gets different figures depending on whether one counts the latter three subjects (hereafter referred to as “cognate disciplines”) as AI. Below, we give figures both for items that fall under the “artificial intelligence” heading alone, and for items that fall under the heading “artificial intelligence” or under the heading of one of the cognate disciplines.

Number of researchers

MAS gives number of authors in CS, AI, and the cognate disciplines of AI, but these figures don’t pick up on the amount of research done as well as publication count figures do.

The IEEE Computational Intelligence Society has ~7,000 members and the IEEE Computer Society has ~85,000 members, so the membership of the first is 8% the membership of the second.

Some other relevant figures (which don’t paint a cohesive picture):

According to the Bureau of Labor Statistics, there are 26,700 computer and information science researchers in the US.

ACM’s Special Interest Group on Artificial Intelligence (SIGAI) has “more than 1,000 members.”

The International Neural Network Society (INNS) has “more than 2,000 members.”

Number of journals

MAS lists 1360 CS journals, with 106 in AI, and 172 in either AI or one of AI’s cognate disciplines, so 8% and 13% respectively.

Publication counts

Between 2005 and 2010, of those publications listed under MAS’s “CS” heading, about 10% were listed under “AI” and about 20% were listed under “AI” or one of its cognate disciplines. One sees roughly the same percentages if one looks at publications between 1990 and 1995, between 1995 and 2000, and between 2000 and 2005. Searching Google Scholar for “Computer Science” and “Artificial Intelligence,” one finds that the number of hits for the latter search is about 30% the number of hits for the former search, which could mean that the amount of AI research is significantly more than 10% the amount of CS research, but some papers that contain the phrase “artificial intelligence” are not artificial intelligence research, and some computer science papers may not contain the phrase “computer science.”

Number of conferences

MAS lists 3,519 “top conferences” in CS and 361 “top conferences” in AI, and the former number is about 10% of the latter number. There are 561 “top conferences” in AI or cognate disciplines, so 16% the number of CS conferences.

Number of organizations

Microsoft Academic Search lists 11,338 organizations for CS and 7,125 organizations for AI, so 63%. If one counts cognate disciplines as AI, the number of AI organizations is 21,802, so 192% that of CS organizations. Taken in isolation, this would suggest that the amount of AI research is much greater than 10%.

“Number of organizations” seems likely to be a weaker metric of amount of research than “number of publications,” etc., so this should be discounted. Nevertheless, the fact that the ratio of AI organizations to CS organizations is so much higher than the other ratios that we looked at is a puzzle. Perhaps the difference comes from the CS community and the AI community having different cultural norms. Or, perhaps MAS is less consistent about how it counts organizations than how it counts publications.

Famous prizes awarded for AI research vs. CS research

ACM Turing Award: Six out of 46 prizes were awarded for AI research, so 13% of the total.

Nevanlinna Prize: One of the 8 prizes was awarded for AI work, so 12.5% of the total. However, the prize for AI work was awarded in 1986, which is a long time ago.

Amount of funding

In 2011, the National Science Foundation (NSF) received $636 million for funding CS research (through CISE). Of this, $169 million went to Information and Intelligent Systems (IIS). IIS has three programs: Cyber-Human Systems (CHS), Information Integration and Informatics (III) and Robust Intelligence (RI). If roughly 1/3 of the funding went to each of these, then $56 million went to Robust Intelligence, so 9% of the total CS funding. (Some CISE funding may have gone to AI work outside of IIS — that is, via ACI, CCF, or CNS — but at a glance, non-IIS AI funding through CISE looks negligible.)

Other major U.S. funding sources for CS research include ONR, DARPA, and several companies (Microsoft, Google, IBM, etc.) but we have not investigated these funding sources yet. We also did not investigate non-U.S. funding sources.

The growth of the AI field

We did not investigate the growth rate of the number of AI researchers in sufficient depth to make meaningful estimates. However, the growth rate of the number of scientists and engineers in all fields might serve as a very weak proxy measure for the growth rates of AI or CS.

For example, the annual growth rate of science and engineering researchers in OECD countries, between 1995 and 2005, appears to be about 3.3%, corresponding to a doubling time of 23 years. This needs to be viewed in juxtaposition with indications that average researcher productivity (as measured by patents per researcher, amount of time spent training per researcher, the number of coauthors per paper, and the number of papers cited) has been decreasing. The NSF Budget for Information and Intelligent Systems (IIS) has generally increased between 4% and 20% per year since 1996, with a one-time percentage boost of 60% in 2003, for a total increase of 530% over the 15 year period between 1996 and 2011. “Robust Intelligence” is one of three program areas covered by this budget. According to MAS, the number of publications in AI grew by 100+% every 5 years between 1965 and 1995, but between 1995 and 2010 it has been growing by about 50% every 5 years. One sees a similar trend in machine learning and pattern recognition.

Notes on further research

Future research on this topic could dig much deeper, and come to more robust conclusions. Our purpose here is to lay some groundwork for future research. With that in mind, here are some miscellaneous notes to future researchers investigating the current size and past growth of the AI field:

If the papers being cited are newer, that could indicate more rapid progress. On the other hand, it could also indicate faddishness, and one would somehow need to differentiate between the two things.

Some citation databases that could be useful for analyzing citation patterns are are Scopus, Web of Science, MS Academic Search, and Science Citation Index (SSI).

Some sources of noise in citation counts are: (a) Journal editors asking authors of submitted papers to add citations to other papers in the same journal in order to boost the journal’s impact factor & (b) Authors citing their own papers excessively in order to increase their citation counts.