There is a lot of discussion this week about a new paper in PNAS (Andregg et al, 2010) that tries to assess the credibility of scientists who have made public declarations about policy directions. This comes from a long tradition of papers (and drafts) where people have tried to assess the state of the 'scientific consensus' (Oreskes, Brown et al, Bray and von Storch, Doran and Zimmerman etc.).

What has bedevilled all these attempts is that since it is very difficult to get scientists to respond to direct questions (response rates for surveys are pitiful), proxy data of some sort or another are often used that may or may not be useful for the specifics of the 'consensus' being tested (which itself is often not clearly defined). Is the test based on agreeing with every word in the IPCC report? Or just the basic science elements? Does it mean adhering to a specific policy option? Or merely stating that 'something' should be done about emissions? Related issues arise from mis-specified or ambiguous survey questions, and from the obvious fact that opinions about climate in general are quite varied and sometimes can't easily be placed in neatly labelled boxes.

Given these methodological issues (and there are others), why do people bother?

The answer lies squarely in the nature of the public 'debate' on climate. For decades, one of the main tools in the arsenal of those seeking to prevent actions to reduce emissions has been to declare the that the science is too uncertain to justify anything. To that end, folks like Fred Singer, Art Robinson, the Cato Institute and the 'Friends' of Science have periodically organised letters and petitions to indicate (or imply) that 'very important scientists' disagree with Kyoto, or the Earth Summit or Copenhagen or the IPCC etc. These are clearly attempts at 'arguments from authority', and like most such attempts, are fallacious and, indeed, misleading.

They are misleading because as anyone with any familiarity with the field knows, the basic consensus is almost universally accepted. That is, the planet is warming, that human activities are contributing to the greenhouse gases in the atmosphere (chiefly, but not exclusively CO 2 ), that these changes are playing a big role in the current warming, and thus, further increases in the levels of GHGs in the atmosphere are very likely to cause further warming which could have serious impacts. You can go to any standard meeting or workshop, browse the abstracts, look at any assessment, ask any of the National Academies etc. and receive the same answer. There are certainly disputes about more detailed or specific issues (as there is in any scientific field), and lots of research continues to improve our quantitative understanding of the system, but the basic issues (as outlined above) are very widely (though not universally) accepted.

It is in response to these attempts to portray the scientific community as fractured and in disagreement, that many people have tried to find quantitative ways to assess the degree of consensus among scientists on the science and, as with this new paper, the degree of credibility and expertise among the signers of various letters advocating policies.

It is completely legitimate to examine the credentials of people making public statements (on any side of any issue) – especially if they make a claim to scientific expertise. It does make a difference if medical advice is being given by a quack or the Surgeon General. The database that Jim Prall has assembled allows anyone to look this expertise up – and since any new source of information is useful, we think this can be generally supported. Prall's database has a number of issues of course, most of them minor but some which might be considered more problematic: it relies on citation statistics, which have well-known problems (though mostly across fields rather than within them), it uses Google Scholar rather than the standard (ISI) citation index, and there are almost certainly some confusions between people with similar names. Different methodologies could be tried – ranking via h-index perhaps – but the as long as small differences are not blown out of proportion, the rankings he comes up with appear reasonable.

So it is now possible to estimate an expertise level associated with any of the various lists and letters that are out there. Note that it is worth distinguishing between letters that have been voluntarily signed and lists that have been gathered with nothing but political point scoring in mind (the Inhofe/Morano list was egregious in its cherry picking of quotes in order to build up its numbers and can't be relied on as an accurate reflection of peoples opinions in any way, and similarly contributing to RealClimate is not a statement about policy preferences!). Additionally, it isn't always clear that every signatory of each letter really believes every point in the statement. For instance, does Lindzen really believe that attribution is impossible unless current changes exceed all known natural variations (implying that nothing could be said unless we got colder than Snowball Earth or warmer than the Cretaceous or sea level rose more than 120 meters….)? We doubt it. But as tests of political preferences, these letters are probably valid indicators.

So, do the climate scientists who have publicly declared that they are 'convinced of the evidence' that emission policies are required have more credentials and expertise than the signers of statements declaring the opposite? Yes. That doesn't demonstrate who's policy prescription is correct of course, and it remains a viable (if somewhat uncommon) position to acknowledge that despite most climate scientists agreeing that there is a problem, one still might not want to do anything about emissions. Does making a list of signers of public statements, or authors of the IPCC reports, constitute a 'delegitimization' of their views? Not in the slightest. If someone's views are widely discounted, it is most likely because of what they have said, not who they sign letters with.

However, any attempt to use political opinions (as opposed to scientific merit) to affect funding, influence academic hiring, launch investigations, or personally harass scientists has no place in a free society – from whichever direction that comes. In this context, we note that once the categorization goes beyond a self-declared policy position, one is on very thin ice because the danger of 'guilt by association'. For instance, one of us (Eric) feels more strongly that some of Prall's classifications in his dataset cross a line (for more on Eric's view, see his comments at Dotearth).

But will this paper add much to the 'there [is/is not] a consensus' argument? Doubtful. People are just too fond of it.

But there really is.