AT FIRST SIGHT the story of renewable energy in the rich world looks like a waste of time and money. Rather than investing in research, governments have spent hundreds of millions of pounds, euros and dollars on subsidising technology that does not yet pay its way. Yet for all the blunders, renewables are on the march. In 2013 global renewable capacity in the power industry worldwide was 1,560 gigawatts (GW), a year-on-year increase of more than 8%. Of that total, hydropower accounted for about 1,000GW, a 4% rise; other renewables went up by nearly 17% to more than 560GW. True, after eight years of continuous increase, the amount invested dropped steeply in 2012 amid uncertainty about future subsidies and investment credits. But thanks to increased efficiency, less money still bought more power.

Measuring progress is tricky: the cost of electricity from new solar systems can vary from $90 to $300 per megawatt hour (MWh). But it is clearly plummeting. In Japan the cost of power produced by residential photovoltaic systems fell by 21% in 2013. As a study for the United Nations Environment Programme notes, a record 39GW of solar photovoltaic capacity was constructed in 2013 at a lesser cost than the 2012 total of 31GW. In the European Union (EU), renewables, despite a 44% fall in investment, made up the largest portion (72%) of new electric generating capacity for the sixth year running.

The clearest sign of health in the renewables market is smoke-clogged China, which in 2013 invested over $56 billion, more than all of Europe, as part of a hurried shift towards clean energy. China’s investment included 16GW of wind power and 13GW of solar. The renewable-power capacity China installed in that year was bigger than its new fossil-fuel and nuclear capacity put together.

Whether or not it represents good value for money in all circumstances at the moment, renewable energy has become a serious part of the energy mix. In 2013 Denmark’s wind turbines provided a third of the country’s energy supply and Spain’s a fifth. Some worries are abating. Though power from solar and wind is intermittent, nature often cancels out the fluctuations: sunny days tend not to be windy, and vice versa.

Both forms of generation have their fans, but solar seems to be pulling ahead of wind. Wind technology is running up against the laws of physics: it is hard to see great new gains in siting, or in the design of bearings and blades. And wind turbines are widely considered unsightly and noisy. Solar panels, by contrast, can be surprisingly attractive. Instead of featuring serried ranks of black rectangles, the latest designs look like glittering autumn leaves captured in glass.

Solar flare

The main reason for the growth in solar energy, though, is innovation, not aesthetics. It comes in two forms. The smaller (accounting for around a tenth of existing solar capacity) is thermal storage, in which sunlight is concentrated as heat, for example in molten salt. That can be used to produce steam for power turbines. After some slack years this form of renewable energy is enjoying a renaissance.

Investment in the second, more widespread form of solar energy, electricity produced by photovoltaic (PV) cells, fell back in 2013 after ten years when average annual growth was around 50%. Yet in the same year total global capacity added in solar electricity exceeded that in wind for the first time. Solar received 53% of the $214 billion invested worldwide in renewable power that year. It still provides only a sliver of the world’s energy, and even by 2020 it will make up just 2% of global electricity supply. But the pace of change is remarkable, with more solar capacity installed since 2010 than in the previous four decades.

Along with worries about pollution from other fuels, the biggest boost to solar—both in the rich and the emerging world—is its plummeting cost. In a recent report on solar electricity the IEA noted that the cost of solar panels had come down by a factor of five in the past six years and the cost of full PV systems, which include other electronics and wiring, by three. The “levelised cost” (the total cost of installing a renewable-energy system divided by its expected energy output over its lifetime) of electricity from decentralised (small-scale) solar PV systems in some markets is “approaching or falling below the variable portion of retail electricity prices”, says the report. The IEA expects the cost of solar panels to halve in the next 20 years. By 2050, it predicts, solar will provide 16% of the world’s electric power, well up from the 11% it forecast in 2010. At times of peak demand in places such as Hawaii, where electricity would otherwise come from oil-fired power stations, solar electricity produced by unsubsidised large installations is already competitive. Sanford C. Bernstein, a research firm, reckons that in the right conditions solar, measured by thermal units produced, is already cheaper than both oil and Asian LNG, despite the recent dip in the oil price.

Paint me a power station

Such forecasts are largely based on existing technologies. New solar technology, known as “third generation”, stacks layers of photovoltaic material to capture a much broader section of the spectrum, including invisible parts such as infra-red. Such cells could be printed from graphene (an ultra-light form of carbon) on a 3D printer. There will no longer be a need for solar panels on rooftops. Instead, any man-made surface could be turned into a solar panel with films and paint. In a pilot project in the Netherlands, solar electricity is being generated by a newly built road. Dieter Helm, the Oxford-based energy expert cited earlier, believes that solar power will become so cheap that energy will no longer be seen as scarce.

Other forms of “distributed” generation which provide power for flexible local use and storage are also coming up fast. Domestic fuel cells, for instance, are common in energy-hungry Japan. Such fuel cells can run off the gas grid. Its pipes, notes David Crane, the boss of NRG, an American power company, are simpler, cheaper and less vulnerable to rough weather than the poles and wires of the electric grid. Households can turn their gas into electricity on the spot. That may end up cheaper and more reliable.

Cost apart, the biggest problem with renewables has always been storing the electricity they produce

Some of that gas could come from waste products instead of fossil sources. America’s oldest brewery, Yuengling in Pennsylvania, has installed a combined-heat-and-power (CHP) plant, fuelled by methane produced from waste, which provides 20% of the brewery’s energy needs. In Ukraine, which is trying to become independent of Russian natural-gas supplies, the European Bank for Reconstruction and Development is financing a 2.25MW biogas plant at a sugar refinery near Kiev. In Britain the first self-powered sewage works came into operation in October 2014, at a saving of £1.3m ($2m) a year. And biogas now accounts for one-tenth of gas consumption in China, where 42m households turn their animal and human waste into methane.

Cost apart, the biggest problem with renewables has always been storing the electricity they produce. That gave a big advantage to incumbent power companies, which could afford large capital investments in generation and storage. For domestic consumers, the power produced from solar panels on the roof is of limited use if they cannot store it, because they still have to buy from the grid in the evening when they need it most. But if intermittent energy can be stored, its economics are dramatically improved: the cost of installing capacity remains the same but the cost per kilowatt hour shrinks.

The easiest storage is someone else’s. In regimes with “net metering” rules, common in some green-minded places including 43 American states, the energy utility is obliged to buy renewable power from small-scale producers at the same price at which it sells its own electricity. That is a startlingly good deal for the producer, less so for the company. But it applies only to small amounts of power and is unlikely to last.

Meanwhile breakthroughs in storage are creating other options. Businesses and households can store cheap, home-generated electricity as thermal energy. An American company sells a device called Ice Bear which makes ice at night with cheap electricity (and in cooler temperatures), then uses it to cool air in the daytime, saving energy and money.

All these technologies are becoming cheaper and more practical, and in some countries are boosted by generous subsidies. Germany rebates 30% (an average of €3,300, or $4,000) on the cost of a solar-plus-battery household system, and offers low-interest credit for the rest. California has legislation in place under which a third of its energy must come from renewable resources by 2020. The state has told its three large utilities to provide 1.3GW of storage capacity. Around 85MW of this is likely to be used by small providers with solar panels.

Bloomberg New Energy Finance (BNEF) has done the sums for a German household planning to install a 5kW solar system and a battery with 3kWh of storage capacity at a cost of around €18,000 ($22,000). The solar panels would cut the household’s power consumption by roughly 30%; adding the storage system could increase the saving to as much as 80%. At the current cost of the equipment, and assuming no rise in electricity prices, the return on the investment would be barely 2%. But on more realistic assumptions—a continuing rise in electricity prices of 2% a year and a big fall in the cost of solar capacity and storage—the rate of return could be a juicy 10% or more.

The Gigafactory, which will build batteries for the Tesla electric-car company, aims to cut the cost of battery storage towards what many see as a crucial benchmark: $100 per kWh against $250 now. That will bring the price of an electric car close to parity with that of a conventional one, maybe even before the end of this decade, hints Elon Musk, the CEO. But better batteries will have other advantages too. One is that electric cars, when not being driven (which is 95% of the time, research suggests), can be used for storage. And batteries that are being displaced by more efficient versions can play a part in domestic storage.

The storage business is booming. Navigant, a consultancy, reckons that in 2014 alone projects amounting to 363MW were announced. BNEF estimates that by 2020 some 11.3GW will be installed, 80% of it in America (chiefly California), Germany, Japan and South Korea, and that investment in storage by then will be running at $5 billion a year.

The biggest advantage of storage is that it dispenses with the most inefficient part of the power industry: the generating capacity that is held in reserve to meet peaks of demand. In the state of New York, for example, one-fifth of the generating capacity runs for less than 250 hours a year. By some counts, a megawatt of storage replaces roughly ten megawatts of such generating capacity—an irresistible saving.

In rich countries new forms of storage and generation are eating away at the model that has sustained the electricity industry since the days of Thomas Edison. In parts of the developing world where there are no incumbents, they offer the best chance for whole populations to get any power at all.