Addendum: At the end of this post I note how Phil Jones finally admits he has lost or overwritten the original raw data from prior CRU products. One of Jones’ admissions is that his group continuously overwrites their data. He tries to say some data has not been ‘adjusted’ or deleted, etc. But the truth is he has reprocessed the data so many times very little of it is still pristine. As I said at the end, this invalidates all claims to date built upon this unverifiable data. Without the ability to reproduce results, then the results cannot be defended. My guess is only 10-20% of the data is untouched. And it is the ‘adjustments’ on the remaining that needs to be audited and cross checked. Going forward there may be some claims to be derived, but nothing in the current record that cannot be reproduced would be considered verified.

Addendum2: What we may be seeing in all this is the slow biasing of ground station data as discovered at NCDC recently. Would it be true irony if the ‘divergence’ Briffa detected in the tree rings was actually all the fudging Jones was doing on the temp record? – end addendum

In a previous post on the ClimateGate 2 emails I noted how Phil Jones, the head of CRU, was a serial deleter and FOIA dodger. I was being too kind.

I decided it would be worthy to paint the evolving picture of how this man lied to people requesting data and – worse – how he lied to his coworkers at UAE and CRU. If anyone is regretting the new release of emails it is Phil Jones (though Michael Mann will come in a close second).

The emails show over time an arrogant ass fool who felt he was too pure to be bothered with criticism, technical challenges and scientific scrutiny. Time and time again he simply made things up and communicated these fantasies as fact. The email record exposes a petty and whiny man whose career should now be truly over.

Let’s begin the story in Feb of 2004 with email 1076336623 between Phil Jones and Tas Van Ommen, who is holding proxy data from Antarctica. In the email Jones and Ommen conspire to deny Steve McIntyre the raw data he requested, even though Tas Van Ommen has the data in hand. The lies spewed by these two disreputable scientists is astonishing:

Ommen to Phil: What you will find below is (in reverse chronological order) an email interchange between Steve McIntyre and myself. He has been asking for LD [Law Dome] data for a while (since your GRL paper came out) and to my chagrin, I have put him off once already, for reasons I spell out below. … Ommen to McIntyre: Dear Stephen, The 18O data used in Mann and Jones 2003 was provided as an advance copy in 2003, and you are welcome to have access to it and it will certainly be placed in public archives. …. It is this next paper that controls the timeframe for release to you and archives. While I should await peer review for a release to the archives, I am happy to pass on a copy of the data set to you on an advance basis as soon as the paper is submitted I expect in a couple of months. Phil to Ommen: Dear Tas, Thanks for the email. Steve McIntyre hasn’t contacted me directly about Law Dome (yet), nor about any of the series used in the 1998 Holocene paper or the 2003 GRL one with Mike. I suspect (hope) that he won’t. I had some emails with him a few years ago when he wanted to get all the station temperature data we use here in CRU. At that time, I hid behind the fact that some of the data had been received from individuals and not directly from Met Services through the Global Telecommunications Service (GTS) or through GCOS.

Keep note of this lie about the Met Services, because Jones will later claim it was National Met Services (NMSs) that were the source and that it was they who were stopping him from providing the raw ground station data.

A year later in Feb 2005 we have email #1107454306 between Mann and Jones. Here is what Jones has to say as be plays the part of a braggart:

The two MMs [McIntyre and McItrick] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.

Jones is really worried about these two ‘skeptics’ since they know a lot more about statistic than poor ol’ Phil. And thus begins a sad saga of deception and delay. The man has lost his mind and now is obsessed with hiding the data – or more accurately the major problems with the data.

By January 2007 (email #1432) the FOIA requests were starting to rattle CRU and now Jones is having to start ‘splaining what the fuss is all about.

I would suggest you contact the Met Office, to get their view on the use of FOIA on this. … The data requested are not on our site or theirs (Met Office). What is there [at CRU], as you’ve found out, is gridded data where we combine the station data, with marine data (which the Hadley Centre, Met Office only have)

Here Phil begins to admit he does not have the raw data anymore to reproduce the ‘gridded data’. It is the processed ‘gridded data’ that needs to be validated in order to determine if the grids are correctly built, which are in turn used to build regional, hemispheric and global temperature anomalies. If this first step is garbage (or biased), then all conclusions based on it are also garbage. Phil knows the raw data is needed to verify the gridded product.

Phil Continues:

I suspect this request is from a climate change skeptic who wishes to try and discredit me, by finding some bad data or bad stations which we likely shouldn’t have used [1]. The law of large numbers, however, means that the average is amazing robust to a few outliers. We have also spent years of effort trying to reduce these outliers to a minimum. It would be extremely difficult for anyone to exactly reproduce what we have with the Met Office jointly produced [2].

Two more hints of honesty from Jones. First he now claims the purpose is to find bad station data. He will repeat this in later emails, exposing the fact the data is not just lost, but in really bad shape. The second fact he exposes is it is nearly impossible to reproduce his past results. Which, per the standards of the scientific method, means the conclusions must be withdrawn. This also affirms the skeptics claims that one cannot simply go to the US data sets and rebuild Jones’ claims.

Finally Jones changes his story as it applies the NMSs:

The data have been collected over many years, and in come cases we have been given data from national met services (NMSs) on the proviso that we do not pass on the raw data to third parties, but we can use them in derived products – such as HadCRUT3. Finally, I’m not sure this data are mine to pass on. I do pass small subsets of the data onto fellow scientists, but never the whole dataset. None of the data are collected by us. We assemble it from what NMSs in all the countries provide over a system called the CLIMATE network.

So Jones now claims he does get the data from the NMSs – completely opposite the claim he gave to Tas Van Ommen. He also notes that he has violated the rule on passing data to third parties before (and he actually has done it many times before). He’ll return to this subject in a gross and pathetic way in 2009.

By March 2007 (email #4341) comments at McIntyre’s Climate Audit site have caused University of East Anglia mamagers to take notice and start taking over the FOIA response battle:

I repeat, I am not primarily interested in the dispute between yourself and ClimateAudit and will keep my opinions to myself. However when I read that people are suggested methods of legal redress against the University for not supplying research data, I felt that I needed to act.

This is from Alan Kendell who had to alert one David Palmer (FOIA Manager) to the true mess Phil Jones had created. The echoes of this concern are captured in the email traffic captured at the end of this email – I suggest everyone read this in its entirety. For example, Phil tries to claim the gridded data should be good enough (even though that is the result people want to verify):

I have told these people on several occasions that some of the data are restricted. They refuse to believe this. We make all the gridded data available on the CRU web site. 99.9% of climate scientists are happy with this, and our data are widely used.

Here Phil is lying to his CRU and UEA colleagues. He is not being open and honest, but he will be in less than a month. Phil whines and moans a lot as the new scrutiny kicks in. My favorite line is how on one hand Jones complains about having to deal with ‘blogs’, but on the other tries to point the powers that be at UEA to an alarmist blog for cover:

Alan, I appreciate your concern about UEA and ENV’s image, but I don’t appreciate you calling our press office about what is happening on the Climate Audit website. The website is run by a self appointed group, who ignore most climate facts. They are not interested in getting at the truth. If you want to learn more about the subject I would suggest the Real Climate website.

One month later, in April 2007 (#email #0314) Phil starts coming clean on what is the real problem – he does not have the data to reproduce his results. This is a response he has worked with David Palmer:

I can’t do anything about B. This is because I don’t have a copy of the station data we had in 1990. The station database evolves and we weren’t able to keep versions of it – as we added, amended and deleted stations. We didn’t have the data storage luxuries we have now. Tell him the best he can do is to use the current version of CRUTEM3(v) or CRUTEM2(v) – as the latter is still available on our web site, though not updated beyond 2005. These latest versions are likely different from what we had in 1990. Australia and China have both released more data since then – it is likely that much of this was not digitized in 1990. He will say the grid resolution is now different, but this is again due to greater disk storage available.

It will take 2 years before this failure is made public. Along the way Jones will spin out variations of these lame and irrelevant excuses. By 2009 – after a year of trying to avoid complying with the law and acting like a real scientist – things begin to move fast. Jones has another flash of openness in a May 2009 email (#4460):

We could decide to make the station data available that will go into CRUTEM4 (and hence HadCRUT4). My issue about not doing it is when will it stop. They will then want programs. As you know there are two key files (the 61-90 normals and the SD file). The station headers within the station data aren’t that well documented. If MOHC does release the station data, I’ll let MOHC deal with all the flak. You don’t know what some of the header info is, for example. There are codes that cause some series to only get used from certain dates. I can barely remember some of it.

Again, we learn something totally damning to the IPCC claims. The station data they still have is also crap. Those problems Jones blamed on 1980s storage capacity are still there – and not due to lack of storage, Jones has no idea what makes up the gridded products nor whether it is any good. Again, under the professional standards of the scientific and engineering processes and methods, the fact the data is garbage means all claims derived from it are now invalid.

Another May 2009 email (#2117) shows Jones collating a series of half truths and excuses. Phil is responding to the Met Office as it works to actually meet their FOIA requirements

2. I have signed agreements with some Met Services (European ones) in the 1990s that I would not pass on their data to third parties. The data could be used in the gridding though and gridded products made available. I never kept a list of which stations these were though, as I never thought such problems would arise. 3. Work on the land station data has been funded by the US Dept of Energy, and I have their agreement that the data needn’t be passed on. I got this in 2007. 4. You web site says that anyone requesting the data should apply to me, so tell him that’s what they should do. I think you should remove this sentence, by the way. It is this that has opened up the issue again. 5. The data aren’t yours to release! Maybe there is no formal IPR agreement, but there is an implicit one. 6. We’ve altered the version that you have anyway. We’re also in the process of doing more of this. 7. You’d need to waste your time combining the two parts of the data and removing the stations that don’t get used.

So let’s review. In ‘2’ Jones claims he has some old agreements (somewhere), but he cannot connect the agreements to the station identifiers. So he basically is saying he never could protect the date rights. In ‘3’ he postulates a complete lie. All US federally funded work is available to anyone as public property. The DoE would never, and could never, make such an assertion (take it from a person who has made a living in federal contracting). At best the contractor (in this case CRU) can make a claim of IPR, but it has to do so at the time and when the data was generated and labeled as such.

In ‘5’ Jones admits he does not really have any signed agreements. He has verbal or imagined agreements. In ‘6’ he admits that CRU to this day does not retain copies (configuration managed) of the data has it is processed. A serious breach of data auditing and another reason his units work is unrepeatable. And finally in ‘7’ he confirms a prime complaints of the skeptics – without knowing which stations to use in developing the gridded data, you cannot replicate CRU results. A major and damning admission.

So the skeptics were right. Jones’ methods are not repeatable, his data is not controlled and maintained and even he cannot replicate his results. In a July 2009 email (#1577) Jones confirms many of these revelations:

Also from looking [at Climate Audit] they will not stop with the data, but will continue to ask for the original unadjusted data (which we don’t have) and then move onto the software used to produce the gridded datasets (the ones we do release). CRU is considered by the climate community as a data centre, but we don’t have any resources to undertake this work. Any work we have done in the past is done on the back of the research grants we get – and has to be well hidden. I’ve discussed this with the main funder (US Dept of Energy) in the past and they are happy about not releasing the original station data.

Phil makes another accidental admission (clearly due to the frustration by him losing the FOIA battle). He states the work is done under research grants and must be hidden. Why? Did they misuse funds? Did thees grants come from US and UK funds – and therefore expose his team to full scrutiny? Note how is latest fear is the code being exposed (which a lot was in Climategate 1 and showed shoddy work). Is there much more here?

In an August 2009 email (#3497) Jones reaches an all time low in rationalizing his actions:

I did send some of the data to a person working with Peter Webster at Georgia Tech. The email wasn’t to PW, but he was in the CC list. I don’t know how McIntyre found out, but I thought this was a personal email. This was one of the first times I’d sent some data to a fellow scientist who wasn’t at the Hadley Centre. As I said I have taken pity on African and Asian PhD students who wanted some temperature and precipitation data for their country. I also don’t see why I should help people, I don’t want to work with and who spend most of their time critisising me.

The man is clearly losing it, The folks at CRU and UEA are trying to work with him to do the right thing and he whines about it.

The excuse making continues in email #0752. Finally the true picture comes to light in October 2009 as Phil is forced to fess up (see email #1192):

Phil’s comment: No one, it seems, cares to read what we put up on the CRU web page. … If we have lost any data it is the following: 1. Station series for sites that in the 1980s we deemed then to be affected by either urban biases or by numerous site moves, that were either not correctable or not worth doing as there were other series in the region. 2. The original data for sites that we adjusted the temperature data [Phil: for known inhomogeneities, or what?] in the 1980s. We still have our adjusted data, of course, and these along with all other sites that didn’t need adjusting. 3. Since the 1980s as colleagues and NMSs [National Meteorological Services] have produced adjusted series for regions and or countries, then we replaced the data we had with the better series.

This is just stunning. He went from pretending to hide data from a scientific critic to admitting he does not have the data. And now we know more about why.

According to ‘1’ he threw out (cherry picked?) data that did not fit their expectations and assumptions. That is clear and obvious. ‘2’ is an admission all he has is the output of his calculations – not the inputs. So not only is the data he filtered out loss (and why he filtered it) what he has has been ‘adjusted’. It is this “selection” and “adjustment” process nf raw data that drives the gridded products. This is what everyone wants to confirm.

‘3’ again indicates he does not even hold copies of the original data, but lets the NMSs overwrite data without any controls.

What a mess. What a liar. All those people who believed Phil Jones was a qualified man of science with a clear audit trail of his work now know he was a shoddy charlatan.

If this had been data on medical trials for drugs, structural testing for foundations, buildings or bridges, or safety data on cars, trains or planes the man would be fired and possibly charged with some form of criminal negligence. But this is climate science, where professional rules of conduct are apparently optional.

Update: More revelations at Climate Audit on Jones and FOIA