PEOPLE who believe that climate change due to human emissions of greenhouse gases is a hugely exaggerated problem tend to put a lot of weight on perceived shortcomings in records of the earth's rising temperature. They find much cause for dissatisfaction both in “proxy” studies of the temperature centuries ago, which seek to measure changes in the climate through their effects on such things as tree rings, and in more recent instrumental studies which use data from thermometers gathered around the world. The fact that the Climatic Research Unit of the University of East Anglia carries out work in both these hot-button areas was one of the reasons why the release of its researchers' e-mails last November was treated as such a big deal.

But the desire for a better surface temperature record is not unique to climate sceptics. Many climate scientists who are fully signed up to the view that the world is warming at an alarming rate—a proposition for which there is a great deal of evidence other than the surface temperature record—also want the record improved. This is why climate scientists, statisticians, meteorologists, metrologists and others will soon be meeting at Britain's Met Office, in Exeter, to discuss an ambitious programme for revamping the information infrastructure which underlies all assessments of climate change on the basis of weather records. The aim is to greatly improve the scope, quality assurance and transparency of such analyses, both by broadening and shoring up the base on which they stand and by widening the capabilities and expertise of the community that undertakes them.

Better data more openly analysed: exactly what climate sceptics (for want of a better word) say that they want and thus should welcome. Given that at least some of them have shown a willingness to roll up their sleeves and get into technical details and debates, indeed to pride themselves on being citizen-scientists, they might even be expected to join in. Such engagement might encourage participants from the sceptic side to accept the good faith of academic scientists (they often don't) while opening the minds of academics to the idea that there are extra muros reserves of critical intellect, not to mention volunteer labour, which could improve their work. Any happy outcome along these lines might prove hard to see, what with the distracting flocks of pigs arcing through the sky on their way to the new ski resorts in hell. But what's life without a little hope.

The Exeter meeting is being put together by a steering committee that includes statisticians and others outside the normal run of climate studies. Its public face, to a certain extent, has been Peter Thorne, until recently at the Met Office, now at the Co-operative Institute for Climate and Satellites in Asheville, North Carolina, a consortium set up to help academia work with the government's National Center for Climatic Data (NCDC). Dr Thorne points out that the three surface-temperature records currently in use—one put together by scientists at the Climatic Research Unit and the Met Office, one by America's National Oceanographic and Atmospheric Administration and one by NASA—provide only monthly averages. They also use data from only a relatively small subset of the total number of weather stations around the world, which accounts in part for their coarse spatial resolution. The end results are good enough for showing long-term trends to the satisfaction of most of the scientific community. But they do not allow researchers—nor, more importantly, planning departments, health-care providers and the like—to pick up on more salient trends such as those of hottest daily temperatures, or most severe rainfall, or whatever.

The purpose of the Exeter meeting is to work out in detail how, as a beginning, to set up a “databank” of results from all the weather stations for which there are records. The NCDC has records for thousands of weather stations, some dating back more than a century. In the more recent past it has amassed records from 6,000 stations which have taken measurements every three hours or so for the past 15 years, and thus might serve as the basis for a far more fine grained assessment of what the climate is doing than the current monthly trends. But there is a lot of climate data the NCDC and other public databases don't have. National weather services often keep their station-by-station records to themselves so that they can use them for various commercial purposes, only making higher level amalgamated products available to the scientific community at large. The quality of those products, and their suitability for climatology, cannot be assessed in the absence of the data on which they are based. Convincing weather services to change their ways on this is one of the biggest challenges that Dr Thorne and his colleagues face. It may require the application of pressure at a government-to-government level, not to mention financial compensation, as well as some public shaming and convincing argument.

The data alone are not enough. Metadata are also necessary—data about how the measurements were made, how the situation of a given weather station has changed or been encroached on, and so on. Anthony Watts, a retired TV meteorologist who runs a popular climate-sceptic blog, has coordinated a network of volunteers documenting apparently poorly sited weather stations for some time at surfacestations.org. The NCDC believes, on the basis of preliminary data and in a way that Mr Watts disputes, that the evidence for poor siting does not affect the overall trends in the surface record. But Dr Thorne says the contribution is important, and more such work would be valuable.

Records made by weather services but not as yet available in digital form will also be required, as might records in private hands that would have to be sought out piecemeal through “search and rescue” missions. To be of real use such metadata need to go beyond qualitative assessments of siting and equipment and be put into a numerical form that machines can read—clerical work which might conceivably be outsourced to volunteers, as some astronomers now outsource the classification of galaxies.

Once all this data were assembled, anyone who had the skills to produce useful climate data sets from them could do so. The algorithms used to create subsets of stations and to adjust for problems in their data would be published openly, and the results of different approaches could be compared. This already happens, to some extent, with the three long-term records of monthly means, which use the data from very similar, though not identical, lists of stations but correct for various perceived drawbacks in the data (such as a lack of coverage in the arctic) in different ways. With a bigger databank and a wider range of data sets assumptions about what differences in processing make a big difference to the results could be checked against each other. Dr Thorne also has plans for generating test data in a format which could show the strengths and weaknesses of different procedures.

A surface temperature databank like this, and the infrastructure around it, would be a big step forward. Dr Thorne does not criticise the current records, but points out gently that both the use of high-powered computing with large data sets and the state of the art in terms of access to data were very different when the current efforts first got under way in the 1980s. It's hard to imagine why people really interested in testing the current records and improving on them—there are some small-scale independent efforts going on in this area—would not be eagerly chipping in with helpful or critical suggestions. However, this is not happening.

The website (surfacetemperatures.org) set up in preparation for the Exeter meeting is hardly a hotbed of activity. On the blog Dr Thorne has set up to allow people to provide feedback on more than a dozen draft white papers, dealing with everything from data interpolation to project governance, it's a rare post that manages to attract as many as two comments. “It's disappointing,” says Ian Jolliffe, a statistician on the Exeter meeting's steering committee. By way of contrast, the top post on Mr Watts's blog at the time of writing, which deals with a new paper on the ever popular topic of the shortcomings of using tree rings and other proxies (a subject on which, as it happens, Dr Jolliffe has made contributions of which sceptics approve), has over 1,000 comments.

This is in part Dr Thorne's own fault. While surfacetemperatures.org has been publicised on a number of mailing lists and the like within the scientific community, Dr Thorne decided not to tout his wares directly to bloggers, on the basis that he would inevitably be seen as playing favourites in some way and polarise the issue unhelpfully. As a result Mr Watts, for one, says that he first became aware of the project when asked questions for this column. Apprised of it, he says that while “a noble effort, it is a reaction to a series of data transparency blunders rather than a proactive approach to open replication”. If Dr Thorne had accepted the risk of making direct contact, he might have established more dialogue, or at least a better record for proactivity.

That said, surfacetemperatures.org has hardly been kept secret. Dr Thorne and a Met Office colleague, Peter Stott, published an article about what they are trying to do in Nature, a journal which bloggers of all sorts read and comment on regularly. Indeed a post by another hand on Mr Watts's blog took issue with a specific aspect of that article—but did not go into the bigger question of what surfacetemperatures.org is trying to do. There have been posts on the subject on a couple of blogs in the past week (see here and here), and there is now one on Mr Watts's blog, but they followed directly from questions asked of the various bloggers during the research for this column.

So, while Dr Thorne and his colleagues try to do something that is both difficult and worthwhile in a way that increases transparency, critics outside the community have to date more or less ignored the opportunity to get involved. This looks like a loss for the scientists. They will be deprived of the benefits of focused criticism, and perhaps also of the opportunities that might come from a broadly supported “crowdsourcing” effort that could help with some of their data handling and quality control problems. When their methods and the architecture of their institutions come in for heated criticism after the fact, as they surely will, “you had your chance to comment earlier—or even to get on board” will not be a response that cuts the rhetorical mustard, nor one that moves things forward. But it will still be true, and if it proves a cause for regret and frustration on one side, it should also bring shame on the other.

(About the image: The Atmospheric Infrared Sounder (AIRS) instrument aboard NASA's Aqua satellite senses temperature using infrared wavelengths. The image shows temperature of the Earth's surface or clouds covering it for the month of April 2003. The scale ranges from -81 degrees Celsius (-114° Fahrenheit) in black/blue to 47° C (116° F) in red. The Intertropical Convergence Zone, an equatorial region of persistent thunderstorms and high, cold clouds is depicted in yellow. Higher latitudes are increasingly obscured by clouds, though some features like the Great Lakes are apparent. Northernmost Europe and Eurasia are completely obscured by clouds, while Antarctica stands out cold and clear at the bottom of the image.)