A CAREFUL OBSERVER might note the chunky double glazing on the elegant windows and the heat pump whirring outside the basement entrance. From the outside the five-storey house in London’s posh Notting Hill district looks like any other. Inside, though, it is full of new technologies that aim to make it a net exporter of power. They exemplify many of the shifts now under way that are making energy cleaner, more plentiful, cheaper to store, easier to distribute and capable of being used more intelligently. The house in Notting Hill is a one-off, paid for by its green multimillionaire owner. But the benefits of recent innovations can be reaped by everybody.

That makes a welcome change from the two issues that have dominated the debate about energy in the past few decades: scarcity and concerns about the environment. Modern life is based on the ubiquitous use of fossil fuels, all of which have big disadvantages. Coal, the cheapest and most abundant, has been the dirtiest, contributing to rising emissions. Oil supplies have been vulnerable to geopolitical shocks and price collusion by producers. Natural gas has mostly come by pipeline—and often with serious political baggage, as in the case of Europe’s dependence on Russia. Nuclear power is beset by political troubles, heightened by public alarm after the accident at Japan’s Fukushima power station in 2011. Renewables such as wind and solar—beneficiaries of lavish subsidies—have so far played a marginal role. The main worries were whether enough energy would be available for power generation, transport, heating, cooling and industry; and if so, whether it would cook the planet.

Now new factors are in play. Technological change has broken the power of the Organisation of the Petroleum Exporting Countries (OPEC) to keep the oil price high. Hydraulic fracturing (“fracking”) and horizontal drilling have turned America into a big oil producer, with 4m barrels a day coming from sources which used to be deemed “unconventional”. The boom in producing oil and gas from shale has yet to spread to other countries. America enjoys some big advantages, such as open spaces, accommodating laws, a well-developed supply chain and abundant finance for risky projects. So far it has refrained from exporting its crude oil or natural gas, but exports of liquefied natural gas (LNG) will start this year. Increased trade in LNG will create a more global gas market and greater resilience of supply, undermining Russia’s pipeline monopoly in Europe. America is already exporting lightly refined oil.

An increase in supply, a surprising resilience in production in troubled places such as Iraq and Libya, and the determination of Saudi Arabia and its Gulf allies not to sacrifice market share in the face of falling demand have led to a spectacular plunge in the oil price, which has fallen by half from its 2014 high. This has dealt a final blow to the notion of “peak oil”. There is no shortage of hydrocarbons in the Earth’s crust, and no sign that mankind is about to reach “peak technology” for extracting them. But the fall has created turmoil in financial markets as energy companies lay off workers and cut or delay investment projects.

The implications are more complicated than the headlines suggest. For a start, low prices do not instantly cause supply curbs or make investment dry up. Even costly projects do not stop pumping when the oil price falls. Fracking is a small-scale business. New projects can be halted quickly and restarted when the price picks up. American frackers are now the world’s swing producers, reacting to price fluctuations in a way that was once the prerogative of the Saudis. On a 15- to 25-year time horizon, today’s slide in the oil price needs to be set against the likely long-term trend. Futures markets are betting that the oil price will be back to $90 per barrel in the early 2020s.

For now, though, low oil prices put money in consumers’ pockets and give a bit of breathing space to governments, making it easier to cut fossil-fuel subsidies (and perhaps even tax carbon emissions). In 2013 some $550 billion was spent on subsidising fossil fuels, a policy of extraordinary wrongheadedness that favours the rich, distorts economies and aggravates pollution.

A bigger question on many minds is the effect of rock-bottom oil prices on the shift towards low-carbon energy. Solar, wind and other renewables have recently benefited from unprecedented investments: an average of $260 billion a year worldwide over the past five years. Long, and wrongly, decried as mere boondoggles, they have begun to show real commercial promise in places as diverse as India, Hawaii, and parts of Africa where the climate is favourable, costs are low and other sources of power are expensive. Renewables capacity is rising even as subsidies are falling. China, for example, has already installed nearly half the 200 gigawatts (GW) of wind power it had been planning for 2020, so it is sharply cutting back the subsidies it introduced in 2009.

But the relationship is not always straightforward. Renewable electricity mainly competes with gas- and coal-fired power stations, not with oil. In North America, low oil prices may, paradoxically, lead to higher natural gas prices. Less fracking means there will be less of the associated gas that is produced along with shale oil. More broadly, much of the support for renewables has been political, and there is little sign that this is changing. Worries about climate change continue to ensure that clean energy enjoys strong political support in many developed countries. Whereas shares in oil companies have in recent months fallen along with the price, the S&P Global Clean Energy Index, which covers the industry’s 30 biggest listed companies, has barely budged.

The economics—and particularly the whopping subsidies of the past decade paid out in countries such as Germany and Britain—remain contested. Solar and wind are intermittent, so they are truly useful only if the power they produce can be stored; otherwise they need back-up capacity, typically from fossil-fuel sources. Dieter Helm, an energy expert at Oxford University, says that subsidies for primitive green technology, such as the current generation of solar panels, have been a “colossal mistake”. It would have been much better, he argues, to invest in proven technologies such as electrical interconnectors (linking Britain and Norway, for example) and support research into new kinds of solar power, such as films that can be applied to any outside surface and technologies that use a wider chunk of the spectrum.

Bits of the green-energy world are wilting under the impact of low oil prices. Some biofuels have become less attractive. The same is true for electric cars, which currently make up less than 1% of America’s light-vehicle fleet. Bloomberg New Energy Finance reckons that with petrol at $3.34 a gallon ($0.87 per litre), that share could rise to 9% by 2020. With petrol at $2.09, it would go up to just 6%. At the same time countries and companies thinking of switching from oil-fired power generation to renewables may reconsider. Saudi Arabia, for example, was planning to invest $110 billion in 41 GW of solar capacity by 2032, but may now want to think again.

Take the long view

Yet the long-term trend is clear. In particular solar electricity, and ways of storing it, are getting ever cheaper and better, as this special report will show. Sanford C. Bernstein, a research firm, sees “global energy deflation” ahead. Most of the investment decisions in the fossil-fuel industry are taken a decade or two ahead. The International Energy Agency (IEA), an intergovernmental organisation often criticised for its focus on fossil fuels, says the world will need to stump up about $23 trillion over the next 20 years to finance continued fossil-fuel extraction, but the prospect of much cheaper solar power and storage capabiliy may put investors off. The story may be not so much what falling oil prices mean for clean energy than what the prospect of clean energy will mean for the oil price.

Old energy industries are changing too. Gas will become more abundant and easier to trade. Even coal, the most widely used and so far most polluting fossil fuel, is not inherently dirty. It does not need to be burned but can be cooked instead to produce methane, which can then be used as a fuel or in petrochemicals. Modern coal-fired plants, though pricey, are far cleaner than the belching monsters of the past. The heat they produce is used, not wasted as in many traditional power plants. The emissions are scrubbed of the oxides (of nitrogen and sulphur) that eat away at bodies and buildings. In some projects—albeit for now on a tiny scale—the CO{-2} is also captured for storage or use. Such improvements could make coal as relatively clean as other fossil fuels, though they make commercial sense only if the rules are tilted in their favour. But if the price of such techniques comes down and the cost of pollution goes up, clean coal could be competitive.

Nuclear power, in theory, is a source of cheap, dependable, constant electricity. In practice it is too costly for private investors to back without government guarantees, and its perceived danger makes it unpopular in some European countries and in Japan. One of several flaws in Germany’s Energiewende—supposedly a big shift to green technology—was the hurried abandonment of the country’s nuclear capacity. Besides, many of the world’s existing nuclear power stations will have to close in the coming two decades. Barring a political shift or a technological breakthrough—perhaps in small, mass-produced nuclear plants—it is hard to see the fortunes of nuclear energy reviving.

Demand for energy is likely to hold up for some time yet, mainly thanks to rapid economic growth in emerging economies. The IEA predicts that over the next 25 years it will rise by 37%. Yet increasing efficiency in energy use and changes in behaviour have meant that the hitherto well-established link between economic growth and energy use is weakening.

More for less

America’s economy, for example, has grown by around 9% since 2007, whereas demand for finished petroleum products has dropped by nearly 11%. In Germany household consumption of electricity is now lower than it was in 1990. Global demand used to rise by 2% a year, but the rate is slowing. Even emissions in China, the world’s largest and dirtiest energy consumer, may peak by 2030, thanks to huge investments in new clean-coal power generation, nuclear and renewable energy and long-distance transmission lines. Simon Daniel, an energy expert, sees two conflicting trends: on one hand greater efficiency, local production and storage, on the other increased consumption from the billions of new devices that will be hooked up to the “internet of things”.

On current form the emissions from oil, gas and coal would, on most models, make it impossible to keep the rise in global temperatures below 2˚C by the year 2100; the most likely outcome would be a 4˚C rise, which has prompted calls for most of the world’s remaining hydrocarbons to be left in the ground. The IEA estimates the investment needed for “decarbonising” future electricity production alone at an astounding $44 trillion. The best hope of avoiding that much warming is a huge increase in energy efficiency.

One big component of that task will be to adapt the existing stock of buildings. Amory Lovins, one of the foremost prophets of energy efficiency and founder of Rocky Mountain Institute, a think-tank and consultancy based in Colorado, believes that the scope for improvement remains huge. He has long preached that proper building design and energy storage can eliminate the need for air-conditioning and space heating in most climates, and illustrates this by growing bananas in his own house, on a windswept mountainside in Colorado where winter temperatures can drop to –44˚C. Eliminating the heating system for his house, he says, saved more money than he spent on insulation and fancy windows. His optimism is slowly winning converts.

Despite all the obstacles, pretty much all the technology the world needs for a clean, green future is already available. As A.T. Kearney, a consultancy, notes in a recent report for the World Energy Council, a think-tank: “Energy-efficiency potentials combined with renewable-energy sources and shale-gas potentials provide an abundance of energy that can be made accessible with currently available technologies.”

Transmission costs for electricity are plunging, thanks to solid-state technology, which makes efficient direct-current circuitry safer and more flexible. Power grids which were previously isolated can now be connected: one audacious plan involves a 700-mile, £4 billion ($6 billion) link between Britain and Iceland. Such projects are costly up front, but offer big long-term savings from cheaper power, better storage and increased resilience.

More effective management of supply and demand also offers scope for big savings, as this special report will show. Sensors can now collect vast amounts of data about energy use, and computer power and algorithms can crunch that information to offer incentives to customers to curb consumption at peak times and increase it when demand is low. At the same time business models which can turn a profit from thrifty energy use are developing, and capital markets are waking up to their potential.

That splendidly energy-efficient house in Notting Hill demonstrates just how much can be done right now, even if it does not yet come cheap. Its owner, Michael Liebreich, founded a business called New Energy Finance, which he sold to Bloomberg, a financial-information company, in 2009. He has spent tens of thousands of pounds on making his home thrifty, resilient and productive.

The house is no stranger to energy revolutions. In 1865 its original builders installed a state-of-the-art delivery and storage system: a coal hole in the pavement, sealed by a handsome cast-iron hatch. Gas and then electric lighting, central heating and hot water came later. But the revolution under its current owner is the biggest yet. Despite the airtight insulation the rooms feel airy. Specially designed chimney cowls suck stale, moist air from the house while a heat exchanger keeps the thermal energy indoors.

The house now requires remarkably little input of energy. Gas and electricity bills for a dwelling of this size would normally run to at least £3,500 ($5,500) a year, but once everything is in place the owner expects not only to spend nothing but to receive a net payment for the electricity he produces. On the roof is a large array of solar panels which deliver two kilowatts (kW) of electricity on sunny days. Another source of power is a 1.5kW fuel cell in the former coal bunker. It runs on gas, with over 80% efficiency—far more than a conventional power station or boiler. The electricity from these two sources powers the household’s (ultra-frugal) domestic appliances and its low-energy lighting, as well as a heat pump (a refrigerator in reverse) that provides underfloor heating. A water tank stores surplus heat. Spare electricity is fed back into the grid.

Mr Liebreich does not claim that his house is easily copied, but he insists that through “thinning mist” the future is visible. “The only things that are inherently costly are the thermodynamic process and resource depletion—for everything else costs have come down, are coming down and will come down in future,” he says. In short, most of the forces changing the energy market are pushing in the right direction.