Few scientists realised that freedom of information laws being introduced in Britain, the US and elsewhere would impinge strongly on their work. But one who did was Dr Phil Jones, the man at the centre of the fallout from the emails stolen from the University of East Anglia. Thanks to his brushes with climate sceptics, he knew that the laws would put new powers in their hands.

The emails reveal repeated and systematic attempts by him and his colleagues to block FoI requests from climate sceptics who wanted access to emails, documents and data. These moves were not only contrary to the spirit of scientific openness, but according to the government body that administers the FoI act were "not dealt with as they should have been under the legislation".

But the emails also reveal deep and understandable frustration among the scientists at the huge amount of time and energy they were being asked to give up to deal with the requests. This was particularly galling as the sceptics making the requests were, in the scientists' eyes, more interested in picking holes in their analyses to suit an anti-global warming agenda, than carrying out research that would advance human knowledge.

Jones foresaw that his arch-inquisitor, the Canadian former minerals prospector and editor of the sceptic blog Climate Audit, Steve McIntyre, would be a thorn in his side. As long ago as 2005, before the incoming legislation had been tested in Britain, Jones was laying out his uncompromising views on protecting "his" data. In a note to the prominent US climate scientist Michael Mann in February that year, he noted that "the two MMs", McIntyre and his co-author the Canadian environmental economist Ross McKitrick, "have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone."

Later, in 2007, Jones told his Chinese-American colleague Wei-Chyung Wang and Thomas Karl, director of the US government's National Climate Data Centre: "Think I've managed to persuade UEA to ignore all further FoIA requests if the people have anything to do with Climate Audit." Later, in December 2008, he wrote in an email to Ben Santer at the Lawrence Livermore National Laboratory in California: "When the FoI requests began here, the FoI person said we had to abide by the requests. It took a couple of half-hour sessions – one at a screen – to convince them otherwise, showing them what CA [Climate Audit, McIntyre's website] was all about. Once they became aware of the type of people we were dealing with, everyone at UEA ... became very supportive."

By and large, the records show, these requests were indeed turned down. Of 105 freedom of information requests to the university concerning the Climatic Research Unit up to December 2009, the university refused 77, accepted six in part, had 11 outstanding, and had only 10 were released in full. One was withdrawn.

In May 2008, CRU received an FoI request from David Holland, an electrical engineer from Northampton, for all emails sent and received by the unit's tree-ring specialist Keith Briffa relating to the IPCC fourth assessment of climate science (AR4), which had been published the year before.

The IPCC archives its formal review exchanges and puts that material online but Holland wanted to see email correspondence between scientists about IPCC text conducted outside that process. Subsequent CRU emails discussed ways of avoiding complying with Holland's request. They decided that some emails had not come via IPCC and so could be ignored as outside the terms of the request, for instance. Jones noted: "If only Holland knew how the process really worked!!"

By 2008, the scientists had become used to dealing with, and usually rebuffing, requests for their data. But this demand for their emails heightened their alarm. Days after receiving the request, Jones sent one of the most damaging emails to emerge from the leak. He asked Mann: "Can you delete any emails you may have had with Keith [Briffa] re AR4? Keith will do likewise. Can you also email Gene [Eugene Wahl, a paleoclimatologist at the National Centre for Atmospheric Research in Boulder, Colorado] and get him to do the same ... We will be getting Caspar [Ammann also from NCAR] to do the same."

This seems to have been the email that persuaded the UK's Information Commissioner's Office – the body that administers the FOI act – its handling of FOI requests was not correct. The deputy information commissioner Graham Smith put out a statement last week which said: "The emails which are now public reveal that Mr Holland's requests under the Freedom of Information Act were not dealt with as they should have been under the legislation. Section 77 of the Freedom of Information Act makes it an offence for public authorities to act so as to prevent intentionally the disclosure of requested information." He said the ICO could not take action over the apparent breach because it occurred more than six months ago.

There was more in a similar vein. That month Jones also wrote to Bradley, saying: "You can delete this attachment [probably Holland's FOI request] if you want. Keep this quiet also but this is the person who is putting FOI requests for all emails Keith and Tim have written and received re Ch 6 of AR4. We think we've found a way around this."

The emailers took the view that, whatever the status of data, personal emails were sacrosanct. As Briffa told Ammann a month later: "Our private inter-collegial discussion is just that – PRIVATE ... submitting to these demands undermines the wider scientific expectation of personal confidentiality ... none of us should submit to these requests." Holland says the emails reveal "a deliberate attempt to destroy info which has been properly requested."

One device for withholding the IPCC emails, revealed in the leaked emails, was to say that IPCC documents were not covered by British law. The University of East Anglia now says that no emails were deleted after this exchange. But seven months later in December 2008, Jones revealed in an email to Santer discussing McIntyre: "If he pays £10 (which he hasn't yet) I am supposed to go through my emails and he can get anything I've written about him. About two months ago I deleted loads of emails, so have very little – if anything at all."

It is not clear that this mass deletion (if indeed it happened) was done to avoid FoI requests. Jones has been quoted elsewhere as saying: "We haven't deleted any emails. I delete my own personal emails a year at a time regardless of subject as I have too many, but the university still has the emails." Indeed so, as it transpired.

In any case, the ICO apparently advised UEA that some requests for information did not have to be granted. Jones wrote to Nasa climatologist Gavin Schmidt in August 2008, "All our FoI officers have been in discussions and are now using the same exceptions not to respond - advice they got from the Information Commissioner."

Through 2008, the debate among the emailers grew about coping with the rising tide of FOI requests. Most saw them as a threat to their work – not because they would uncover fraud, but because they took up their time.

Schmidt, one of the hosts of the RealClimate website, wrote consolingly to Santer in December 2008 about dealing with McIntyre: "There are two very different things going on here. One is technical and related to the actual science ... the second is political. The second is the issue here ... Whatever you say, it will still be presented as you hiding data. The contrarians have found that there is actually no limit to what they can ask people for (raw data, intermediate steps, additional calculations, residuals, sensitivity calculations, all the code, a workable version of the code on any platform etc) and like Somali pirates they have found that once someone has paid up, they can always shake them down again."

Others wanted to give some ground. The Stanford University climatologist Dr Stephen Schneider, who runs the journal Climate Change, wrote a round-robin to scientists in January 2009 in which he agreed that "this continuing pattern of harassment ... in the name of due diligence is in my view an attempt to create a fishing expedition to find minor glitches or unexplained bits of code – which exist in nearly all our kinds of complex work – and then assert that the entire result is thus suspect."

But Schneider argued that researchers should give enough data and information on their sources and methods so that those "who are scientifically capable can do their own brand of replication work, but that does not extend to personal computer codes with all their undocumented sub-routines etc." Even so, he felt "it would be odious requirement [sic] to have scientists document every line of [computer] code so outsiders could just apply them instantly. Not only is this an intellectual property issue, but it would dramatically reduce our productivity since we are not in the business of producing software products for general consumption and have no resources to do so."

Presciently, he added: "Good luck with this, and expect more of it as we get closer to international climate policy actions. We are witnessing the 'contrarian battle of the bulge' now and expect that all weapons will be used."

In retrospect, it was clear that things were coming to a head by 2009. Freedom of Information requests were piling up. The scientists were increasingly angered at how long it was taking to fend them off. Let alone what they regarded as the nightmare prospect of having to deliver the data being requested. And, no doubt, the further scientific questions that would arise once the sceptics dug their teeth into the data. As the scientists resisted, anger grew among their critics.

At the end of August 2009, an amateur sceptic called Rupert Wyndham spotted that earlier in the year Jones had been made a fellow of the American Geophysical Union, which published many of his papers. He assembled an international group of sceptics from ten countries and wrote to the AGU's top atmospheric scientist, Alan Robock, to complain. He accused Jones of a range of data crimes. "Honouring a man who consistently breaches the fundamental protocols of scientific method casts a stain on the reputation of the AGU," they wrote. Signatories included Sonja Boehmer-Christiansen, editor of Energy and Environment, Jones's least favourite journal, and Martin Durkin, the British TV producer notorious for his programme The Great Global Warming Swindle.

Meanwhile stories began to circulate outside the university about how CRU was resisting legitimate requests from McIntyre. In early July 2009, when I asked Jones about this, he told me: "McIntyre has no interest in deriving his own global temperature series. He just wants to pick holes in those who do. He wants not only the original station data, but details of all the adjustments we have made over the years. It's just time-wasting." But Jones didn't know what was about to hit him.

The day after the rejection of his demand for the station data, McIntyre announced that a "mole" had sent him a full set of the station data. He published some, from Lund in Sweden between 1753 and 1773 – "sensitive information indeed," he noted on his Climate Audit blog. The following day he claimed on the blog that the mole had been identified. Later McIntyre admitted there was no mole and he had simply found the material.

According to a subsequent article in Nature, McIntyre had stumbled on "ftp" files containing station data that was intended to be shared only by CRU's partners at the Met Office. CRU immediately removed the data from its website, leading to charges from McIntyre that they were engaged in a "purge".

Meanwhile, according to Nature's climate blogger Olive Heffernan, "between 24 and 29 July, CRU received 58 FOIA requests from McIntyre and people affiliated with Climate Audit ... The Met Office, which receives a cleaned-up version of the raw data from CRU, has received ten requests of its own."

With the threat of a "mole" in their midst, climate scientists outside CRU grew wary that their correspondence was not as secure as they might like. In September 2009 Jonathan Overpeck of Arizona University warned colleagues in an email: "Please write all emails as though they will be made public."

In early July, McIntyre appealed against being refused the station data. But it was turned down by the university's director of information services Jonathan Colam-French in a letter dated Friday 13 November, which McIntyre says he received on the 18th.

McIntyre says the timing may be significant here. The first attempt to put online the file containing the CRU emails happened on the morning of Tuesday the 17th. It contained emails up to the 12th. McIntyre says he believes this shows the leak was probably an "inside job" by an aggrieved employee or student at the university angry about the secrecy over CRU's data.

Whoever carried out the hack, there is a tragic irony for Jones and UEA buried in Jones's 2005 correspondence with Mann over the potential for a UK FOI act in which he flagged up what a useful tool it would be for the sceptics. Advising Mann on how to avoid a security breach involving sensitive data that was left unprotected on an ftp server Jones wrote: "Don't leave stuff lying around on ftp sites – you never know who is trawling them."