For decades fossil fuel majors tried to fight the consensus – just as big tobacco once disputed that smoking kills

In 1998 a public relations consultant called Joe Walker wrote to the American Petroleum Institute (API), a trade association representing major fossil fuel companies, with a proposed solution to a big problem.

In December the previous year, the UN had adopted the Kyoto protocol, an international treaty that committed signatory countries to reducing their greenhouse gas emissions in order to avert catastrophic climate breakdown.

Reducing emissions represented a direct threat to the profits of fossil fuel companies and the API was working on an industry response.

“As promised, attached is the Global Climate Science Communications Plan that we developed during our workshop last Friday,” Walker wrote. The workshop had involved senior executives from fossil fuel companies, including the oil multinationals Exxon – later part of ExxonMobil – and Chevron, and the gas and coal utility Southern Company, and a handful of rightwing thinktanks.

Q&A What is the polluters project? Show Hide The Guardian has collaborated with leading scientists and NGOs to expose, with exclusive data, investigations and analysis, the fossil fuel companies that are perpetuating the climate crisis – some of which have accelerated their extraction of coal, oil and gas even as the devastating impact on the planet and humanity was becoming clear.

The investigation has involved more than 20 Guardian journalists working across the world for the past six months. The project focuses on what the companies have extracted from the ground, and the subsequent emissions they are responsible for, since 1965. The analysis, undertaken by Richard Heede at the Climate Accountability Institute, calculates how much carbon is emitted throughout the supply chain, from extraction to use by consumers. Heede said: "The fact that consumers combust the fuels to carbon dioxide, water, heat and pollutants does not absolve the fossil fuel companies from responsibility for knowingly perpetuating the carbon era and accelerating the climate crisis toward the existential threat it has now become." One aim of the project is to move the focus of debate from individual responsibilities to power structures – so our reporters also examined the financial and lobbying structures that let fossil fuel firms keep growing, and discovered which elected politicians were voting for change. Another aim of the project is to press governments and corporations to close the gap between ambitious long-term promises and lacklustre short-term action. The UN says the coming decade is crucial if the world is to avoid the most catastrophic consequences of global heating. Reining in our dependence on fossil fuels and dramatically accelerating the transition to renewable energy has never been more urgent.

Walker outlined a vision of a comprehensive, international campaign to change public opinion on the climate crisis by casting doubt on the scientific research, presenting it as unreliable when the overwhelming majority of scientists had reached consensus.

The communications plan involved finding sympathetic scientists, identifying thinktanks to fund that would produce helpful reports, and working through supposed grassroots groups to hold debates questioning the consensus on global heating, along with a constant flow of media briefings manufacturing uncertainty.

The plan sounded much like a 1960s PR campaign devised by the tobacco industry to delay controls by questioning the science showing that smoking killed. Some of the people involved were in fact tobacco campaign veterans.

The fossil fuel industry had been making use of its lobbying group, the Global Climate Coalition, since 1989 to stress the uncertainties of climate science. But by the late 1990s companies such as BP and Shell were beginning to withdraw from it as public doubt about the problem became increasingly untenable in the face of the evidence.

“Project goal: a majority of the American public, including industry leadership, recognises that significant uncertainties exist in climate science,” the 1998 Walker API memo began. A series of strategic goals was elaborated. It said “victory will be achieved when … recognition of uncertainties becomes part of the conventional wisdom” and “those promoting the Kyoto treaty on the basis of extant science appear to be out of touch with reality”.

Play Video 3:46 Why we need political action to tackle the oil, coal and gas companies - video explainer

After the memo was leaked to the New York Times, the industry said the plan was only a proposal and was never put into effect.

Climate campaigners such as Greenpeace say they believe a highly organised effort by the fossil fuel industry to question climate science, involving scientists and some thinktanks in receipt of fossil fuel industry funding, nevertheless succeeded in the following years in shifting public opinion away from urgent action.

In 2010 the American sociologists Riley Dunlap and Aaron McCright identified conservative thinktanks, along with US conservative politicians, media and fossil fuel corporations, as crucial components in a “denial machine” that emerged in the 1990s.

The activity of this machine would peak when the industry’s financial interests came under threat, most notably in the years after 2007 and the election of Barack Obama, who had pledged to regulate and cap emissions.

Half a century of dither and denial – a climate crisis timeline Read more

Robert Brulle, a professor of sociology and environmental science at Drexel University in Pennsylvania, published the first peer-reviewed study in 2013 of who was funding what he called the climate change counter-movement that delayed action on the crisis. He found that between 2003 and 2010 more than $500m had been donated by private conservative philanthropic foundations to organisations whose output included material disputing the consensus.

Thinktanks, trade associations and front groups were a key part of the effort, he concluded, with their major funders including foundations affiliated to the fossil fuel magnates the Koch brothers, ExxonMobil, and the ultra-conservative Scaife and Bradley foundations.

Brulle also found evidence of a trend to conceal the sources of funding once campaign groups such as the Union of Concerned Scientists, Greenpeace and the Climate Disinformation Database started tracking what they called dark money to climate denial from the mid-2000s.

In the second half of that decade, Koch, Scaife, Bradley and ExxonMobil foundation funding to organisations involved in propagating doubt declined while donations to the same organisations via two anonymised vehicles, the Donors Trust and Donors Capital Fund, increased rapidly.

Among the thinktanks most identified with spreading doubt are the Competitive Enterprise Institute, the Marshall Institute (which folded in 2015), the Cato Institute, the Heartland Institute, the Heritage Foundation and the campaign group Americans for Prosperity.

Elsewhere the Institute of Economic Affairs and the Global Warming Policy Foundation have been prominent publishers of material questioning the consensus on climate science in the UK. These organisations fiercely dispute that any of their work constitutes organised climate change denial.

Richmond v Chevron: the California city taking on its most powerful polluter Read more

Americans for Prosperity, which has received a very substantial part of its funding from the Kochs, helped make resistance to action on climate a feature of Tea Party rallies in the US.

The counter-movement against action wound up to fever pitch in 2009 when it looked as though Obama and the US would sign up to UN climate protocols after the Copenhagen summit due at the end of that year.

Before the summit, individual independent climate experts found themselves subject to devastating attacks. Scientists at the University of East Anglia’s prestigious Climate Research Unit had their emails hacked. The contents of the emails were circulated, with the information they contained having been extracted in a way that suggested the scientists had manipulated their data. A police investigation failed to establish who the hackers were.

The rightwing media labelled it “climategate” and several thinktanks promoted the story enthusiastically. Multiple inquiries would later exonerate the scientists but by then the damage was done; the public’s faith in climate science had been measurably dented.

Half a decade later some fossil fuel industry funding of climate contrarian science was exposed, when Greenpeace found out via freedom of information requests that a prominent academic at the Harvard-Smithsonian Center for Astrophysics, Willie Soon, had attracted more than $1.2m in payments over 14 years from ExxonMobil, Southern Company, the API and a Koch foundation, to the centre for his work. Soon doubted the scientific consensus that emissions were the principal cause of global heating.

He is now an affiliate of the Heartland Institute. Soon strenuously denied that his industry funders had any influence over his conclusions and the Heartland Institute said he was not even aware of who some of the donors to the centre were, making a conflict of interest impossible.

Facebook Twitter Pinterest US college students protest against the Kyoto treaty in support of George W Bush at a climate summit in Bonn in 2001. Photograph: Frank Augstein/AP

There has been a noticeable moderation of views from those previously involved in questioning the science of climate change. Several now acknowledge global heating is caused by human activity but have shifted focus to arguing that the market and technological innovation rather than government action or international treaties curbing emissions are the best ways to tackle it.

The director of the Competitive Enterprise Institute’s (CEI) Center for Energy and Environment, Myron Ebell, for example, told the Guardian: “CEI believes strongly that the policies being proposed by climate alarmists to deal with global warming pose much greater threats to human flourishing than do the effects of global warming. Abundant, affordable energy is a necessary condition of human wellbeing but the global energy-rationing policies being pursued, like those in the Paris climate treaty, threaten to consign billions of people around the world to energy poverty and perpetual economic stagnation.”

ExxonMobil, Chevron, Southern Company and the API all said they recognised the seriousness of the climate crisis and the need for business, governments and consumers to reduce greenhouse gas emissions. The API said the industry as a whole had invested billions of dollars in zero- and low-carbon technologies. Chevron and Southern Company said they had ambitious targets to reduce their carbon footprints. ExxonMobil said its position on climate science in the past had been misrepresented, and that claims regarding what it knew and when had been debunked. None of the companies responded to questions on the communications plan and funding of organisations whose output included doubting the science.

The Koch, Scaife and Bradley Foundations Donors Trust, Donors Capital Fund and Americans for Prosperity did not respond to requests for comment.

The thinktanks said the criticisms levelled at them by climate activists and critics seriously mischaracterised their positions. They said the views they published were those of individual affiliates rather than institutional ones. They added that they respected their donors’ privacy but the source of their money did not influence their research or output, which was completely independent. The Heritage Foundation said allegations it had denied climate science were “seriously inaccurate”. It accepted “the climate is changing, the planet is warming and that humans are playing a role”. Instead it described itself as “sceptics of climate catastrophism and costly policies that will drive energy prices higher”.

The Cato Institute said it had never been in the business of “promoting climate science denial”; it did not dispute human activity’s impact on the climate, but believed it was minimal.