Brace yourself! Gas prices below $3 a gallon can't last.

That's the warning from the International Energy Agency Wednesday. It says plunging oil prices will damage the U.S. shale oil boom and cause supply problems down the road.

Oil prices have dropped by 30% in the past four months, putting oil producers under pressure.

The low prices could deter investment in production, which will eventually hurt supply, the agency's chief economist Fatih Birol said.

Related: Low oil prices could crush Vladimir Putin

In its latest outlook, the IEA did say that lower oil prices could could help oil importing countries and their economies and even lead to increased demand.

But higher prices were needed to ensure future energy security.

The IEA said U.S. oil production would "level off in the 2020s and then start to gradually decline as it will become less economically attractive."

Warning for shale oil

The shale industry could come under pressure even earlier than that.

Deutsche Bank said recently that 40% of U.S. shale oil production scheduled for 2015 would be "uneconomic" if prices drop below $80 a barrel.

The shale oil boom in the U.S. and low prices are masking other risks for consumers, the IEA said.

Related: These countries are getting killed by cheap oil

The world is relying on a small number of producers, it warned, pointing to the uncertainty caused by the turmoil in the Middle East and the crisis in Ukraine.

"The global energy system is in danger of falling short of the hopes and expectations placed upon it," the report said.