The past 10 years have been a rollercoaster for energy markets with the price of oil swinging from $10 (£5) a barrel to nearly $120, triggering demands to explore in areas formerly written off as too expensive or too environmentally sensitive such as Canada's tar sands and the Arctic.

The ratcheting up in the cost of oil and gas should have driven up inflation and led to a switch to alternative fuel sources. Instead, oil companies have recorded the highest ever profits in corporate history as demand continues to outstrip supply. In the process, the industry has reconsidered its strategy to plan for an era of high prices.

It was completely different 10 years ago when Opec, the oil producers' cartel, raised output in reaction to a swift increase in prices, just as Asia was plunged into a currency crisis. The price of crude tumbled to $10 a barrel and such was the gloom about the future that it was not long before one well respected magazine - The Economist - was predicting that oil prices could in future be halved to $5 a barrel.

The near bust in commodity prices saw a transformation in the structure of the world's largest oil groups such as BP, ExxonMobil and Total as they took advantage of the downturn to go on the acquisition trail. The slump encouraged technological innovation with companies using 3-D seismic data to reduce drilling risk, directional and horizontal drilling, plus the injection of carbon dioxide and other measures to improve reservoir recovery.

By 2000, oil prices had recovered to $30 but even an industry veteran such as Sir Mark Moody-Stuart, then chairman of Shell, was pessimistic about the longer-term direction of crude values, insisting his company would use $14 as a future guide.

"In the longer term, technology will increase production capacity and tend to drive the oil price somewhere below $20 a barrel," he said that October. Towards the end of the following year, prices had indeed dipped again, hit this time by a combination of increased Opec output, a big recovery in Russian production and weakness in the US economy.

The 2003 invasion of Iraq by US and British forces caused a price rise and uncertainty in other petroleum producing parts of the world such as Venezuela and Nigeria kept pushing them further.

In October 2004, crude values had reached $50 a barrel and industry analysts began to note the soaring demand from China as it embarked on its most rapid industrialisation process. Within six months, the Wall Street investment bank Goldman Sachs invited ridicule by suggesting the world would see a "super-spike of $105 a barrel".

It explained: "We believe oil markets may have entered the early stages of what we have referred to as a 'super spike' - a multi-year trading band of oil prices high enough to meaningfully reduce energy consumption and recreate a spare capacity cushion only after which will lower energy prices return."

In September 2005, prices rose above $60 a barrel after Hurricane Katrina damaged oil production facilities in the US. The market increased again to $66 a barrel in January 2006 as a result of violence in Nigeria and Iran's conflict with the west, before reaching a new peak in August when oil rose to $75 a barrel after Israel invaded Lebanon. Over the past 12 months the price of oil has almost doubled to nearly $120.

Different reasons have been put forward to explain the rises - from tensions between Washington and Iran, worries about "peak oil", insufficient refinery capacity and under-production by Opec. Certainly, many hedge funds have turned to commodities as equity values have slumped and financial buyers are purchasing crude as a hedge against further falls in the dollar.

The new world of $100-plus oil has encouraged all the big companies to raise the level at which they are prepared to invest in new oilfields and prompted them to re-examine areas previously considered too expensive given the likely size of any find - such as the North Sea. Canada's tar sands - where oil is extracted in an energy-intensive way - has seen a new Klondyke rush with Shell and BP both moving in.

Similarly, oil companies have started to talk openly about the potential that exists beneath the Arctic, despite pressure from environmental groups. The western oil companies have found host nations such as Venezuela and Russia gradually seizing back assets while state-owned oil groups from China and India prowl the world for new supplies.

The growth in the need for oil - particularly in China and India - has led to shortages of rigs and drilling staff. Inflation might have remained relatively muted so far, but costs in the oil sector are surging at 15% a year in some areas. The boom has extended to the service companies that provide all the equipment needed by the oil groups. Having been at the sharp end of downward cost pressures from the larger oil and gas firms in the 1990s, which saw some driven out of business, they have found themselves in the driving seat as demand escalated dramatically.

Oil companies are being forced to pay more than $500,000 a day for deep-sea drilling rigs and service companies have become targets for takeover by private equity firms. Expro International, a Reading-based well-testing specialist, has just received a £1.6bn takeover offer from Candover in partnership with Goldman Sachs.