Dataset construction focusing around individuals

CC research and media coverage have grown steadily over the last quarter century, drawing on a range of actors from the scientific, public, media, and policy domains36. According to MC project37 data reported in Fig. 1, the term “climate change” is currently used in approximately 104 media article sentences per week, roughly 100 times as much as the term “climate skeptic”, a broad term that collectively refers to contrarians and denialists, and also conventional scientific skeptics who are driven by more legitimate motives for dissent28,42. For this reason, we focus on a select set of contrarians who have publicly and repeatedly demonstrated their adamant counterposition on CC issues12—as extensively documented by the DeSmog project (DeSmogblog.com), a longstanding effort to document climate disinformation efforts associated with numerous contrarian institutions and individual actors.

Fig. 1 Growth of research output and media production relating to climate change (CC). a Number P(t) of CC publication indexed by Web of Science by year t, with 93% published after 2000. The approximate linearity on linear-log axes indicates exponential growth with notable upticks in the late 1980s and late 2000s. b The prominence of three CC-related terms, measured by the number of sentences (per week) across all media articles indexed by Media Cloud. (c) Total number of media articles indexed by Media Cloud (per month, across all media sources). Exponential smoothing is used in b and c to tame the noise at the week and month resolution Full size image

The entry point for our large data-driven analysis is to construct a comprehensive list of adamant contrarians, which we achieved by merging multiple data sources. To be specific, we combined three overlapping sets of names obtained from publicly available sources. The first source is the list of past keynote speakers at Heartland ICCC conferences from 2008 to present; the second is the list of lead authors of the 2015 Nongovernmental International Panel on Climate Change (NIPCC) report; and the third is the list of individuals profiled by the DeSmog project. All together, we constructed a list of 386 prominent contrarians, comprised of academics, scientists, politicians, and business people who are primarily anglophone.

We then collected ∼200,000 CC research articles from the WOS database, from which we selected the 386 highest cited scientists (denoted by CCSs). These prominent scientists, many are pre-eminent CC experts with distinguished careers spanning several decades, serve as a size-balanced comparison group. Supplementary Fig. 1 lists the 100 most highly cited CCSs in our sample. We provide more detailed information on the derivation of the CCC and CCS groups in the “Methods” section and Supplementary Information Note 1.

Arriving at two lists of prominent CC actors, we then downloaded article metadata covering roughly ∼100,000 CC media articles from the MC project37, a public database of media article metadata (see “Methods”), which facilitates large-scale data-driven studies at the intersection of society, politics, and media43. For each member (hereafter distinguished by the index i) of the CCSs and CCCs, we counted how many media articles (given by M i ) and in which media sources did he/she receive visibility (hereafter we distinguish media sources by the index s). It is important to note that MC data does not include social media posts (e.g., from Facebook and Twitter) and thus represents publicly visible hybrid content (web-only and dual print-web content) from a wide range of content producers, reflecting varying levels of production effort and quality. We also accounted for article multiplicity, i.e., articles within the MC dataset with different hyperlinks and unique article MC identifiers but with the same title and media source (see “Methods” for more details on this MC article disambiguation procedure).

Figure 2 shows the 100 most prominent CCCs and CCSs in the media, as well as the 100 most prominent media sources. Visual comparison of the rank-ordered plots indicates significant media visibility variation both within and between the two groups. The frequency distribution P(M i ) plotted in Supplementary Fig. 2a, b further illustrates the within-group variation, which is significantly right-skewed. For the CCCs, the average (median) visibility is 104 (22.5) articles; similarly for the CCSs, the average (median) visibility is 57.5 (5) articles. In contrast to these characteristic levels, the most visible individuals within each group have M i in excess of 103 articles. In what follows, we also leverage this within-group variation, specifically by normalizing each individual’s media visibility by their scientific achievements. This effort demonstrates the robustness of the selection criteria used to identify these two groups and the resulting compositional differences between the two groups. This broad variation points to generic success mechanisms observed in various other social systems, whereby previous achievements facilitate new opportunities, visibility, and reputation growth44. Yet such cumulative amplification mechanisms cannot fully explain how non-scientific experts are able to compete with scientific experts in the attention economy facilitated by the media.

Fig. 2 Prominent climate change contrarians (CCCs) and scientists (CCSs) in the media. a The 100 most-prominent CCCs in the media, ranked according to the number of Media Cloud (MC) articles; although all full names were obtained from publicly available lists, we anonymized CCC names to foster privacy. The color scale associated with each CCC indicates the fraction f i of his/her articles that appear in the select-30 most prominent mainstream sources. b The 100 most-prolific media sources among all CCC articles. M s denotes the total number of articles for a given media source, tallied across the pooled set of CCC articles. The sources colored magenta are members of the select-30 media source group. c The 100 most-prominent climate change scientists (CCSs) in the media, ranked according to the number of MC articles. d The 100 most-prolific media sources among all CCS articles Full size image

Authority in the scientific literature

We begin our comparative analysis by measuring group-level contributions to the CC literature in peer-reviewed scientific journals, namely, those that meet WOS rigorous indexing standards. In order to associate individual CCC and CCS with research articles, we match individuals’ names to the list of coauthors associated with each WOS publication, using a tested method to address the author name disambiguation problem45. Accordingly, we find that only 224 of the 386 CCC have a single publication in our WOS dataset. Thus, in our analysis drawing on scientific publication data, we proceed by comparing just the 224 published CCC with a size-balanced subset comprised of the 224 most-cited CCS. We denote these two subsets by 224CCC and 224CCS, respectively.

Proceeding at the group level, we tallied the total number of unique publications (i.e., counting a publication coauthored by two or more members of a given group only once). Figure 3a shows the disparity in scientific productivity, with the 224CCC subset publishing 3367 scientific articles (on average 15 articles per individual). Conversely, the 224CCS subset published 12,665 scientific articles, roughly 3.8 times more than the 224CCC.

Fig. 3 Discrepancy in scientific authority and media visibility—group level. a Total number of publications by the climate change contrarian (CCC; red) and climate change scientist (CCS; blue) groups. 224CCC indicates the subset of 224 CCCs comprised of just the individuals with at least one Web of Science publication; 224CCS indicates the 224 most-cited CCSs. b Total number of citations from the publications in a. Total number of unique media articles from c all media sources and d 30 select mainstream media sources Full size image

Likewise, we tallied the total citations received by each publication set. Figure 3b shows an even larger disparity in citation impact, with 224CCS collecting roughly 7.6 times as many citations (992,206) as the 224CCC (130,833). We analyzed the degree to which this difference is larger (or possibly smaller) than what could be obtained by random chance by performing a random bootstrap sampling of the underlying productivity and citation distributions. Our simulation results show that the disparities are robust to statistical fluctuations arising from finite sample sizes and further demonstrate that the 224CCC productivity and citation impact tallies are indistinguishable from a group of CCC (see Supplementary Fig. 3).

Moreover, from a methodological perspective, the citation tally for 224CCC is likely to suffer from generous overestimation relative to the 224CCS tally (see “Methods” for further detail). To be specific, because leading researchers tend to have net citation tallies in the range of 103–105 WOS citations44,46, i.e., orders of magnitude greater than the citations accrued by the average papers in their field, the misattribution error associated with name ambiguity only marginally increases the citation tally C i for elite scientists belonging to the 224CCS group; contrariwise, misattribution error could significantly increase C i for the majority of the 224CCC group members. Thus the 660% difference in group-wise citations is a lower-limit estimation of the disparity in scientific authority between these CCS and CCC.

Visibility in the media

We continue by comparing group-level media visibility across a large collection of digital and print media CC articles collected from the MC project37. Much like the WOS publication data, which is derived from various journals, our media article data is derived from a wide range of media sources—including print newspapers and magazines, as well as online media (e.g., online news sites, personal and society blogs). Figure 2 shows the most prominent individuals and media sources associated with the CCC and CCS groups, respectively.

We performed two group-level comparisons, one using all media articles and one using a subset of media articles coming from 30 select media sources (see Supplementary Fig. 4 and Supplementary Table 1). These select-30 media sources account for 11% (11,233 articles) of the total media articles analyzed. Because the select-30 media source set was chosen manually, we also compared article counts per group using media source groups identified by an automatic clustering algorithm, which yields consistent results (see Supplementary Fig. 5).

Tallying across all media sources, we count 26,072 articles for CCCs, roughly 49% more than the 17,530 articles associated with CCSs (see Fig. 3c). Tallying across just the select-30 sources, we obtain nearly equal counts: 2482 articles for CCCs and 2463 articles for CCSs, corresponding to just a 0.77% excess for CCCs (see Fig. 3d). Upon further inspection, we found that these count differences also strongly depend on the underlying group composition—in addition to the composition of the underlying media sources.

Repeating the comparison using just the subsets of 224 academically oriented individuals, we obtain a negligible difference of just 1% (16,670 articles for 224CCC and 15,896 for 224CCS). Thus conditioning on either visibility in the mainstream media or on visibility by academically oriented individuals yields parity. However, proceeding with the comparison conditioned simultaneously on select-30 sources and academically oriented individuals reveals a 38% media visibility advantage in favor of the elite scientists (1619 articles for 224CCC and 2235 for 224CCS). These results highlight the nuances associated with comparing groups comprised of individuals with fundamentally different professional orientations. Yet even in this latter and most relevant case, where we compare 224CCC and 224CCS in the mainstream media, there still remains a remarkable discrepancy in the scientific authority and media visibility between these more academically oriented scientists and contrarians.

To further distinguish visibility in the mainstream media, as opposed to blogs and other new media sources, we calculated the fraction f i of articles associated with each individual published by each media source belonging to the select-30 media source group. The color gradient in Fig. 2a, c indicates the value of f i for the most prominent individuals, revealing how contrarian visibility from mainstream media sources is more concentrated on a relatively small CCC subset. To facilitate group-level comparison, we also calculated the distribution P(f i ) for individuals with M i ≥ 10 articles (in order to eliminate individuals with large f i due to small sample size fluctuations). Comparison indicates that, among these more prominent individuals, the average CCS has roughly twice the mainstream prominence as the average CCC; the distribution P(f i ) for the CCSs is also more right-skewed than for the CCCs, see Supplementary Fig. 2c, d. While these results may appear to be inconsistent with the group-wise totals shown in Fig. 3c, d, this apparent discrepancy arises from the fact that multiple CCSs and CCCs can be associated with the same MC article.

Together, these results show that CCCs derive a comparative visibility advantage from non-scientists gaining attention in peripheral non-mainstream media sources. Conversely, the observed parity between CCCs and CCSs in mainstream media sources may reflect media writers seeking journalistic balance when reporting on CC. Indeed, we find that every select-30 media source has provided CCC significant visibility, thereby increasing CCC authority and credibility (see Supplementary Fig. 4). The disproportionate visibility of CCCs, even in mainstream media sources, is reminiscent of early contrarian efforts that leveraged the U.S. Federal Communications Commission Fairness Doctrine to obtain equal press time6. While this policy was officially discontinued in 1987, journalists may still be using it to justify mentioning and interviewing counterpositions when writing on contentious issues such as CC. Indeed, communication scholars have noted that, in the case of CC, such disproportionate visibility—or false balancing—is likely to mislead public perception, suggesting falsely that within the scientific community there is parity in the number of scientists who do and do not agree on the fundamental issues of anthropogenic CC6,8,9,10,11,12.

Juxtaposing authority and visibility at the individual level

To test whether the discrepancy in scientific authority and media visibility is also present at the individual level, and not the result of just a few outliers driving group totals, we disaggregated the WOS and MC data into individual profiles. Figure 4a compares the article count M i between individuals of the same rank within their respective groups. CCCs are consistently more visible in the media relative to their CCS counterparts; this disparity persists even when comparing visibility within the prominent select-30 media sources.

Fig. 4 Discrepancy in scientific authority and media visibility—individual level. a Individuals ranked by their number of media articles, M i . (right) M i calculated using select-30 mainstream media sources only. b Scatter plot of individuals comparing Web of Science publications P i versus media visibility M i ; point size is proportional to the log of total citations, ln C i . c Probability distribution P(r p ) of media visibility per publication, r p ≡ M i /P i ; vertical dashed line indicates the distribution median. d Probability distribution P(r c ) of visibility per citation impact, r c Full size image

Figure 4b shows the total number of media articles (M i ), the total number of WOS publications (P i ), and the total number of citations (C i ). Shown together, this representation highlights the relatively small intersection between the two groups. Despite CCCs holding advantage in gross media visibility, just a few CCCs are on par with the scientific achievement of career experts. Moreover, the scatter plot indicates that CCCs are more likely to have larger M i values than their CCS counterparts within the same P range. Thus, despite the selection criteria that explicitly gives CCSs the advantage in the scientific domain, the discrepancy between the two groups is manifestly prominent.

To further emphasize this point, we also calculated the visibility per unit of scientific achievement, thereby accounting for compositional differences between the two groups at the individual level. The ratio r p,i ≡ M i /P i measures the number of media articles per publication for each individual. Similarly, \(r_{c,i} \equiv M_i/\sqrt {1 + C_i}\) measures the media articles per citation impact, where the square root is used to adjust for the skew in C i , while C i +1 avoids the singularity for individuals with C i = 0. Figure 4c, d show the probability distributions, P(r p ) and P(r c ), which convey the markedly different ranges and concentrations of r between the 224CCC and 224CCS. For both measures, the mean (respectively median) r value calculated for 224CCC is ∼15 (∼40) times larger than the mean (median) value for 224CCS. However, distributions calculated using ratio values scaled by the group average (e.g., \(\tilde{r}_{p,i} \equiv r_{p,i}/\langle {r_p} \rangle\)) indicate a common distribution shown in Supplementary Fig. 3d, e.

This distribution scaling result indicates that the mean media visibility per scientific authority is an appropriate group-level indicator. As such, if we use the 162 CCSs not included in the 224CCS group as an alternative comparison group, one that is comprised of scientists who are not nearly as elite as the 224CCS, we still find that the 〈r〉 values for this alternative group are on par with the average values for 224CCS: The group mean values are 〈r p 〉 = 15.4 (224CCC), 1.04 (224CCS), 1.66 (162CCS); 〈r c 〉 = 18.8 (224CCC), 0.94 (224CCS), 0.47 (162CCS). Thus, by controlling for individual-level variation in scientific authority, we show that, even compared to a less prominent scientific comparison group, the 224CCC still have remarkably high media visibility per scientific authority.

Classification of how individuals are sourced in the media

In order to identify how CCCs and CCSs obtain media visibility, we analyzed the full-text content of 2256 media articles produced by 6 mainstream media sources: the Guardian, New York Times (NYT), Washington Post (WP), FOX News (FOX), LA Times (LAT), and the Wall Street Journal (WSJ). For each article, we located individuals’ names and inferred the context associated with their sourcing, which we annotated according to five types (see “Methods” for further details). In order to facilitate comparison, we further grouped these five types into two broad categories: mentioned, representing a passive sourcing; and contributed, representing a more active sourcing.

Figure 5a shows the frequency of each sourcing type by media source, revealing mentioned as the most common sourcing type. The main exception is FOX, which tends to quote individuals on non-scientific grounds, which is also fairly common in the WP. Another common sourcing are quotes containing scientific content, which are more common for CCSs, as CCCs are rarely associated with this sourcing type. The least common sourcing types observed are non-scientific quotes and adversarial quotes, with most instances of these types associated with CCCs. Notably, the Guardian, NYT, and WSJ featured the most number of articles authored by CCSs.

Fig. 5 How climate change contrarians (CCCs) and climate change scientists (CCSs) are sourced in CC articles—by media source. a Frequency distribution showing how individuals are sourced in media articles according to five types separated into two categories: mentioned (purple) and contributed (green). Pie-chart insets: The outer ring indicates the fraction of individuals analyzed by group; the inner ring indicates the fraction of articles analyzed featuring individuals from just CCCs (red arc), just CCSs (blue arc), or both groups (black arc, with this percent value indicated at the center). b The frequency of three sourcing configurations for unbalanced articles featuring either CCCs or CCSs but not both. The black segment indicates the frequency of articles featuring mentioned and contributing individuals. c The two most common sourcing configurations for balanced articles featuring both CCCs and CCSs. For example, articles with CCCs and CCSs mentioned is the most frequent configuration for The Guardian, which occurred in 44% of the articles featuring both CCCs and CCSs; the second most frequent configuration featured CCCs contributing and CCSs being mentioned (20%) Full size image

By focusing our content analysis on individuals, we are able to estimate the frequency of cross-group articles—those articles that source both CCCs and CCSs. Our analysis indicates cross-group articles to be around ∼7% for each source, with the exception of the WSJ, which featured both groups in 12% of its articles. These percentages are likely to be a lower-bound estimate to the frequency of balanced sourcing of individuals from each group within the same article, since it is also possible that individuals not included in our select sets of 386 individuals were also mentioned or contributed to these articles. As a result, the most common configuration we observed features just CCCs or just CCSs but not members of both groups. Figure 5b indicates the relative frequency of the three possible sourcing configurations, showing that articles featuring just CCCs (respectively just CCSs) are those with individuals classified as mentioned (respectively contributed); the exceptions are FOX and WSJ, which instead most commonly feature CCCs as contributing sources. This is consistent with previous analysis of FOX, which observed a greater ratio of CC doubters to believers among those interviewed, as compared to CNN and MSNBC47.

A common theme in the CC communication literature is false balance, representing how the journalistic tradition of balancing sources across opposing views gives rise in the case of CC to an inaccurate representation, one that falsely suggests that there is a balanced debate between equally sized groups8,9,11,12. Figure 5c provides insight into this phenomena by showing the two most common configurations for the subset of articles featuring both CCCs and CCSs. Our results show that the most common motif among articles sourcing CCCs and CCSs are those that are also balanced by sourcing type, with the exception of the WSJ, which instead tends to include CCC contributions juxtaposed by CCS mentions. Among the sources that balance according to sourcing type, the Guardian, NYT, and LAT most commonly mention individuals from each group, whereas WP and FOX tend to incorporate individual contributions from each group.

Co-visibility in the media

It was unfeasible to apply the content analysis to the entire dataset, and so we turn to network analysis to identify additional relational patterns of co-visibility within groups and across their media interface. To proceed, we first merged the sets of CCS and CCC media articles. Whereas M i counts the total number of media articles for individual i, the co-visibility M ij ≤ M i counts the number of articles that feature both individuals i and j. Combining the matrix elements M ij calculated for all pairs of individuals, we construct the symmetric co-visibility matrix M. Supplementary Fig. 6 shows M calculated in two ways: using all media sources and using just the select-30 media sources. Note that individuals with M i = 0 are not included in the matrix M; and for an individual with M i > 0, if they do not appear in any media articles with any other individuals, then they are also not included in M.

Visual inspection of the co-visibility matrix reveals two fundamental features. First, CCCs are more prominent than CCSs when two or more individuals are featured in the same media article: 58% of individuals who have appeared in a media article with another individual are CCCs; considering co-visibility within select-30 media sources, this visibility advantage grows to 62%. Second, the strongest co-visibility (largest M ij ) are within group rather than between group (see Supplementary Note 2 and Supplementary Fig. 6), reflecting the results of our content analysis.

To illustrate this latter point, we applied the Louvain modularity maximizing algorithm48 to cluster the co-visibility matrix into communities of individuals. To be specific, we applied this unsupervised algorithm to identify groups of individuals who are more connected to other individuals within their community than without. Figure 6 uses a network visualization layout in which communities are indicated by each network spine, revealing a three-community structure. Moreover, we ordered the individuals (nodes) along each spine according to their network centrality (using the PageRank metric), such that the most prominent individuals within each community are located toward the apex. Inspection of the composition of each community reveals two types: two are mixed and the third is primarily composed of CCCs—a clear example of an archetypal echo chamber.

Fig. 6 Media article co-visibility network—individual level. Clustered representation of the co-visibility network: nodes are climate change contrarians (CCCs) and climate change scientists (CCSs) who have at least one media article associated with at least one other individual. Links are colored according to three types: links between members of the CCC (CCS) group are red (blue) and links between groups are black; the percentages of links by type are reported in parentheses (e.g., 52% of links are within the CCC group). We used a modularity-maximizing clustering algorithm48 to identify three communities, with nodes ordered along each spine according to its network centrality—as such, the most prominent individuals are located at the apex. Two communities are well mixed, whereas the third represents an extremely polarized echo chamber comprised primarily of CCC. (inset) Magnification of the apex showing the most prominent individuals Full size image

Asymmetric flow of citations within the CC citation network

We also analyzed the organizational patterns recorded in the WOS citation network. Citation networks reconstructed from the reference lists of publications provide insight into the evolution of the scientific endeavor—a complex system emerging from the interactions between researchers, scholarly outputs, collective knowledge, and emergent culture49. Scientific authority, which emerges from the repeated interactions of individuals within the community of active scientists, can thus be inferred from citation totals at varying levels of aggregation44. In the present context, distinct citations represent quantifiable interactions between individuals, likely ranging from attribution, to critique, to outright dismissal. This latter type of negative citation occurs relatively frequently50, reflecting the oppositional nature of debate around contentious scientific issues51.

In this way, we used the CC citation network to assess the flow of authority between the two research-oriented subsets, 224CCC and 224CCS, at both the group and individual level. We start at the group level, using the ∼50,000 other CC scientists who were not members of either the 224CCC or 224CCS groups as an external self-consistent comparison group. Figure 7a shows the proportion of CC scientific article citations flowing between the 224CCC, 224CCS, and CC Other groups, with 224CCS having 17 times the citation authority as 224CCC (20.2% of the total citations from CC Others are directed toward 224CCS, whereas 1.1% are directed toward 224CCC). In direct comparison, the 224CCC cited 224CCS twice as often as in reverse. Even after normalizing citation rates by group productivity, we find that 224CCC cite 224CCS 20 times more frequently than in reverse and that 224CCS receive 79% more citations than they produce (see Supplementary Fig. 3a).

Fig. 7 Research article citation network—group and individual level. a Within-group and between-group citation flow as a percentage of the total number of citations produced across three researcher groups. Node size captures the net citation flow into a given group; link width is proportional to the fraction of the total citation flow, with link color indicating the source group. For example, 20.2% of the total citations are directed toward 224 climate change scientists (CCS) (corresponding to 0.44% of the total 50,442 researchers analyzed in the group-level citation analysis), whereas only 1.1% are directed toward the 224 published climate change contrarians (CCC); roughly 17 times as many citations flow from the CC Other to CCSs as from the CC Other to CCCs. b Nodes in the network are CCS and CCC researchers with at least one publication receiving at least one citation from another node (i.e., connected within the citation network); roughly 90% of the nodes are CCSs because 218 CCCs have no publications citing or cited by other publications within the set of publications by CCCs and CCSs. The links capture the total number of citations flowing from publications authored by scientist i to the publications authored by scientist j and are colored according to the source node; gray links are de-emphasized using low opacity level. Node size is proportional to the log of the total edge weight (citations) entering a given node. We used the Louvain modularity-maximization method48 to identify groups of nodes belonging to a particular community—i.e., groups of node that are more connected to other nodes within the cluster than without. These communities are plotted along each of the spines, with nodes ordered according their size, so that the most prominent individuals are located at the center. Each community contains several CCCs, located mostly at the peripheral (low-prominence) tips, with just a few exceptions. Word clouds show the 50 most frequent Web of Science publication keywords associated with each community; keyword size is proportional to the log frequency Full size image

We also analyzed the directed citation flow between any given pair of individuals that occurs when a publication p a authored by individual a cites a publication p b authored by individual b. To be specific, we counted the citation linkages a ∼ p a → p b ∼ b that connect any pair of authors, where a ∼ p a indicates that individual a is an author of publication p a , and p a → p b indicates that publication p a cited p b .

Figure 7b shows the resulting interpersonal CC citation network, comprised only of scientists who gave or received at least one citation within our WOS dataset. As such, there are only 168 CCCs connected within this citation network. We again used the Louvain algorithm48 to identify groups of nodes that are densely connected together, representative of coarse research communities. Our results show five communities, each represented as a spine with individuals ranked according to centrality, with the most authoritative individuals located at the apex. Represented as such, the network of scientific authority shows that the majority of CCC are located toward the periphery. Interestingly, the peripheral CCC within each community appear to direct most of their citations toward the most prominent CCS, possibly representing adversarial interaction in the form of negative citations aimed at discrediting their research findings50.