Einstein probably didn’t say “insanity is doing the same thing over and over again and expecting a different result”, but might have done had he kept tabs on the impact of the annual climate negotiations. We have now had 25 of them – the 26th Conference of the Parties (Cop26) will take place in Glasgow in November – yet greenhouse gas concentrations continue to rise inexorably. Cops – especially the one in Paris in 2015 – have changed attitudes and influenced policy. But to meet the goal of reducing global emissions to net zero by mid-century we have to start doing something substantial and different in addition to seeking stronger commitments at Cop26.

The good news is that there are now real grounds for optimism that we can slow and ultimately stop greenhouse gas emissions. Renewable energy currently outcompetes fossil fuels in many areas and continues to become cheaper every year. New energy storage options, ranging from cheaper batteries to green ammonia, are emerging. New ways to produce proteins at scale without destroying rainforests are being developed.

When these solutions become so good, and so cheap, that they routinely outcompete their fossil fuel and biodiversity-destroying counterparts, greenhouse gas emissions will decline to near zero. Getting there, though, needs some serious focus on green technologies supported by policies that will get them rolled out.

Q&A What is Cop – and how will it help? Show For almost three decades, world governments have met every year to forge a global response to the climate emergency. Under the 1992 United Nations Framework Convention on Climate Change, every country on earth is treaty-bound to “avoid dangerous climate change”, and find ways to reduce greenhouse gas emissions globally in an equitable way. Cop stands for conference of the parties under the UNFCCC. The UK will host Cop26 this November in Glasgow. In the Paris agreement of 2015, all governments agreed for the first time to limit global heating to no more than 2C above pre-industrial levels, and set out non-binding national targets on greenhouse gases to achieve that. However, these targets are insufficient, and if allowed to stand would lead to an estimated 3C of heating, which scientists say would spell disaster. For that reason, the Cop26 talks in Glasgow are viewed as the last chance for global cooperation on the emergency, with countries expected to come with tough new targets on emissions. The negotiations will be led by environment ministers and civil servants, aided by UN officials. Nearly every country is expected to send a voting representative at the level of environment secretary or equivalent, and the big economies will have extensive delegations.

Each of the 196 nations on earth, bar a few failed states, is a signatory to the UNFCCC foundation treaty. The Cops, for all their flaws, are the only forum on the climate crisis in which the opinions and concerns of the poorest country carry equal weight to that of the biggest economies, such as the US and China. Agreement can only come by consensus, which gives Cop decisions global authority. Fiona Harvey Environment correspondent

A research group in Oxford, led by my co-authors and myself has been identifying groups of “unicorn” technologies, named by analogy with “unicorn” startups valued at over $1bn, that can each deliver a reduction of at least a billion tonnes of CO 2 a year. Unicorns don’t depend on fundamental new discoveries so should be ready for large-scale deployment within a decade. With a moderate investment in research and development or other support, they could in combination unlock multiple benefits across whole “energy ecosystems”.

One example of such an ecosystem centres on converting cheap renewable energy – either from sunny places with few people, or spare wind from the North Sea – into hydrogen and ammonia, to provide zero-carbon fuel for shipping, heavy goods vehicles and even potentially aviation. They could provide the energy storage needed to complement variable wind and solar energy, process heat for industry, and turn iron ore into steel. The technologies that would unlock this ecosystem are the next generation of high-efficiency solar cells, low-cost electrolysers, and hydrogen and ammonia fuel cells and engines. All are proven technologies that simply need to be cheaper – further development could rapidly unlock a virtuous circle of falling costs and increased deployment.

They would allow least-developed and developed nations to share ambitions and interests in technologies where they have comparative advantages, such as solar resources, land or skills

Another example is exceptionally water-efficient plants that can grow in semi-arid areas or on degraded land – prickly pear, pineapples and euphorbias are examples. Such plants could sequester carbon in soil on a large scale, and be fermented to proteins in place of soya beans, whose growth causes widespread deforestation and huge greenhouse gas emissions; they could even be used to make plastics. The unicorn technologies in this case include learning how to grow plants that we have historically ignored, along with precision separation and fermentation of the biomass.

Getting a bold, unanimous agreement to ratchet up global ambition to act on climate at Cop26 will be very hard at this stage of the game, with the usual cabal of petro-states and coal producers doing their utmost to dilute any decisions of consequence. However, an approach focused around “coalitions of the willing” in the bottom-up spirit of the Paris climate agreement, could enable substantial progress without being stymied by vested interests.

Each of these coalitions, built from nation states but also regions and even corporates, could focus on one or more of the unicorns. They would support research and development, advance demonstrators, or provide lower-cost finance or performance guarantees to encourage rollout and help the technologies become cheaper. They would allow least-developed and developed nations to share ambitions and interests in technologies where they have comparative advantages, such as solar resources, land or skills. Management and licensing of intellectual property could encourage coalitions to form and accelerate technology deployment.

‘Management and licensing of intellectual property could encourage coalitions to form and accelerate technology deployment.’ Benban Solar Park in Aswan, Egypt. Photograph: Khaled Elfiqi/EPA

The formation of these coalitions could be seen as evolution of an earlier UK initiative called Mission Innovation. It targeted sectors or problems, rather than – as we are proposing – selected technology areas that underwrite whole energy ecosystems, which we believe would be a very much more productive approach.

This should not be seen as “picking winners”, but rather as “picking runners”. Funders should be prepared to see failures as well as successes.

As co-chair with Italy of Cop26, the UK has an excellent opportunity to build coalitions that would lead to cheaper alternatives to fossil fuels. The UK is home to several companies and world experts in both the “hydrammonia” economy and semi-arid plants, and British businesses could be leading players in these two examples of ecosystems.

In parallel with preparing for Cop26, the government should also change the UK’s broad approach. Neither academia nor industry on their own has the principal objective of solving the climate crisis. They have neither skills nor funding to develop technologies that are (nearly) out of the laboratory, build demonstrators when necessary, and create the technical and financial conditions needed to enable large-scale rollout. We believe the UK needs something like the Advanced Research Projects Agency-Energy, created in the US, which “advances high-potential, high-impact energy technologies that are too early for private-sector investment”.

By fostering coalitions of the willing, which would focus on clusters of unicorn technologies, the UK could make a major contribution to solving the climate crisis. Cop26 could indeed be a turning point – but it cannot simply be more of the same.

• Mike Mason is a fellow at the Smith School of Enterprise and the Environment, University of Oxford. This piece was written in collaboration with Cameron Hepburn, director of the Smith School, and Chris Llewellyn Smith, of the department of physics