The businesses controlling the future of the global climate are names most people have never heard of. State-owned companies with rights over the exploitation of national fossil fuel reserves now account for a majority of oil and gas produced around the world, overtaking publicly listed companies such as ExxonMobil, BP and Shell.

But most of these 71 state-controlled companies – with a few exceptions, such as Norway’s Equinor – are remarkable for their secrecy, their lack of accountability to any but a small cadre of top government officials, and their absence from globally coordinated attempts to tackle the climate emergency.

Q&A What is the polluters project? Show Hide The Guardian has collaborated with leading scientists and NGOs to expose, with exclusive data, investigations and analysis, the fossil fuel companies that are perpetuating the climate crisis – some of which have accelerated their extraction of coal, oil and gas even as the devastating impact on the planet and humanity was becoming clear.

The investigation has involved more than 20 Guardian journalists working across the world for the past six months. The project focuses on what the companies have extracted from the ground, and the subsequent emissions they are responsible for, since 1965. The analysis, undertaken by Richard Heede at the Climate Accountability Institute, calculates how much carbon is emitted throughout the supply chain, from extraction to use by consumers. Heede said: "The fact that consumers combust the fuels to carbon dioxide, water, heat and pollutants does not absolve the fossil fuel companies from responsibility for knowingly perpetuating the carbon era and accelerating the climate crisis toward the existential threat it has now become." One aim of the project is to move the focus of debate from individual responsibilities to power structures – so our reporters also examined the financial and lobbying structures that let fossil fuel firms keep growing, and discovered which elected politicians were voting for change. Another aim of the project is to press governments and corporations to close the gap between ambitious long-term promises and lacklustre short-term action. The UN says the coming decade is crucial if the world is to avoid the most catastrophic consequences of global heating. Reining in our dependence on fossil fuels and dramatically accelerating the transition to renewable energy has never been more urgent.

Of the world’s 20 biggest emitters, according to Richard Heede’s research, 12 are national oil companies (NOCs): state-owned or state-controlled entities set up to exploit a country’s fossil fuels, mostly oil but also gas and coal.

Three of the five biggest emitters are NOCs: Saudi Arabia’s Aramco is the biggest of all, Russia’s Gazprom is in third place, and the National Iranian Oil Company fifth.

Researchers said a lack of reliable historical data on Chinese coal companies – and ambiguous ownership structures – prevented the inclusion of Chinese coal production by either state- or investor-owned companies, although they are a major source of global carbon emissions.

In recent years, the scramble by national governments to take control of their own reserves and the profits from them has spelt a bonanza for NOCs, which hold an estimated 90% of known reserves.

According to the International Energy Agency, the biggest NOCs last year produced 84m barrels a day of oil and gas. That compares with 21m from the seven biggest publicly listed oil companies and 67m from other sources, including smaller independent oil companies and specialists.

Play Video 3:46 Why we need political action to tackle the oil, coal and gas companies - video explainer

In 2017, the latest year for which figures are available, NOCs’ assets were at least $3tn, though probably far higher, and at least 25 national governments derived more than a fifth of their income from NOCs.

That the general public know little about NOCs is by design: most do not publicly report on their operations in the way demanded of companies listed on stock exchanges, their meetings take place behind closed doors and their strategies are often opaque.

The Natural Resource Governance Institute published the first comprehensive global database of NOCs this spring, after more than three years of painstaking research. Patrick Heller, the lead author, says less than one in three NOCs published enough information to fulfil the “key indicators” he identified as most important for citizens to assess how well their NOCs were performing for the public good.

“And some of the world’s most important NOCs continue to publish almost nothing,” he adds.

When it comes to future strategy and climate change, few NOCs give any indication of engagement. The Guardian asked 12 of the biggest how their operations fitted with their countries’ commitments under the Paris climate accord, and what they planned to do to transition to a net-zero carbon global economy by 2050, but none answered the questions.

Petrobras alone made any response, citing achievements in cutting emissions from its operations, investment in cleaner technology and future goals to reduce emissions per unit of fossil fuel produced by 32% in oil exploration and production, and 16% in refining by 2025.

“The fact that many NOCs remain so opaque has major implications for how well their countries respond to climate change,” says Heller. “It is still difficult for regulators to get reliable and regular information on how much they are spending, future production projections or fundamental business strategies. This makes it challenging for the public to confidently assess the viability of national climate commitments or pursue a transition to a cleaner energy mix. It also makes it hard to assess the risks that the money NOCs are spending on exploration today could end up in stranded assets that won’t be financially viable as the world shifts away from fossil fuels.”

Governments and NOCs are inseparable in terms of their interests and behaviour in climate-related negotiations, says Bryony Worthington, the executive director of Environmental Defense Fund Europe and a cross-bench peer, who has observed NOCs over the past two decades in international climate talks. “There is no distance between NOCs and their governments in negotiations – they are the same,” she reports. And many of them – most publicly Saudi Arabia – “have acted as a brake on progress”.

Key to understanding – and changing – NOCs are the mandates given them by their government owners, says Christoph Frei, the outgoing secretary general of the World Energy Council. “Most NOCs are mandated only to extract oil and gas,” he says. “They do not have strategies in place to fulfil the Paris agreement.”

To bring pressure to bear on big oil companies, activists have deployed a wide variety of tactics, from pickets of conferences and action on oil rigs, to shareholder activism such as divestment campaigns.

But many countries with NOCs are not democracies, or have spotted human rights records: from Saudi Arabia and Russia to China and Venezuela, protesters face draconian laws and police hostility.

At a minimum, more transparency is essential. “We should also be looking for mandate change,” adds Frei, to force the managers of NOCs to take account of their responsibility for climate chaos. He believes a global carbon price would be the most effective way to force NOCs to recognise their damage to the climate.

Change will be hard, though not impossible, concludes Heller. “But the degree to which these companies have grown up on major oil rents, and the political power that has gone along with them, pose a significant obstacle to efforts to transform them.”