The link between commodity speculation and global food price weirdness just got stronger, and researchers warn that a new and potentially calamitous price bubble is imminent.

Previous research had blamed a combination of food-to-fuel crop conversion and financial speculation for driving food prices up in sudden, unpredictable spikes, independent of changes in supply or demand. But the market model used in those findings only explained trends retroactively. It didn't test a prediction.

"People could say, 'You must have fit the model to the data.' Now we have this additional evidence, and people can see that we didn't," said Yaneer Bar-Yam, president of the New England Complex Systems Institute and co-author of the new study, which was released March 6 on arXiv. "This confirms that the model is very solid in its underpinnings. It says our understanding of what's going on is right."

Since 2007, global food prices have surged twice, each time rising by more than 50 percent in less than a year, putting people in poor countries at risk and sparking social unrest around the world. Economists have argued over what drove the surges: Maybe it was bad weather, or changing dietary habits, or changing farming practices, or even a change in how global food markets now work.

In earlier work Bar-Yam's group used mathematical models to test links between food prices and proposed explanations for market behavior. They identified two culprits: replacing food crops with biofuel crops seemed to be driving food prices slowly upward, while commodity speculation – investors betting on food prices – appeared to cause the spikes.

The link to speculation was especially significant. For most of the 20th century, only farmers and food industry companies speculated on food. It helped them offset risks posed by fluctuating prices and production levels, and generally had a stabilizing effect on markets. But in the late 1990s, financial industry-led deregulation efforts opened food speculation to hedge funds, investment banks and other people with only betting interests.

Some economists said the massive rise in speculation had no effect on food markets, but others argued that speculation – especially over the past several years, when stock market volatility sent investors into commodity markets – now made food prices volatile and prone to surges. That's precisely what Bar-Yam's group originally found, but with an important methodological caveat.

Their model was tested by matching its output against the market's recent history, but data from the market's recent history also guided the model's design. As a result, it was possible that the model didn't really explain food price trends, but merely imitated them with the benefit of hindsight. Whether it predicted the future was an open question.

In the new study, predictions made by the researchers' original model are compared to actual food prices between March 2011 and January 2012. Placed on a graph, the lines match closely, and do so despite spanning a major change in price trends at the last bubble's peak.

Global food prices (blue dotted line) track with the original model (green dotted line) and a slightly tweaked version of that model (red dotted line) after March 2011 (blue vertical line). Image: Bar Yam et al./arXiv

"If you have a straight line, extend it and say, 'Aren't we predictive,' it doesn't give that much confidence," said Bar-Yam. "If it changes direction, that's a much more severe test of what's happening."

Both the European Union and United States are now considering whether and how to limit commodity speculation. In the U.S., such limits are required by the Dodd-Frank Act, but have been fiercely resisted by the financial industry.

It's expected that the U.S. Commodity Futures Trading Commission will enact speculation limits by the end of 2012, though they might still be blocked in court. But even if the rules pass, they're arguably weak, focusing on "position limits," or caps on the maximum number of contracts a single speculator can hold. The rules won't won't prevent markets from being overwhelmed by speculation.

In their ideal form, commodity markets should contain "70 percent commercial hedgers and 30 percent speculators. The speculators are there to provide liquidity. In the summer of 2008, it was discovered that it's now 70 percent speculation and 30 percent commercial," said Michael Greenberger, former director of the CFTC's Division of Trading and Markets. "Now reports are coming out that it's 85 percent speculation and 15 percent commercial. You have markets dominated by people with no real interest in the economics of supply and demand, but who are taking advantage of bets authored by Wall Street that prices will go up."

Greenberger said the CFTC's current position limit rules would likely bring speculators' share of food markets down to 60 percent, which is "still a volatile market that wouldn't truly reflect supply and demand."

In the absence of stronger rules, direct Congressional action is an option. A speculator-driven rise in gas prices has made crude oil speculation a potential target of regulation, and agricultural commodities would likely be covered by the same rules.

According to Bar-Yam, the model predicts another speculative surge by 2012's end, with food prices rising above their 2011 peaks. "Unless they get speculators under control, eventually there's going to be a big bubble," he said. "It's a countdown to a real problem."

Citation: "The Food Crises: Predictive validation of a quantitative model of food prices including speculators and ethanol conversion." By Marco Lagi, Yavni Bar-Yam, Karla Z. Bertrand and Yaneer Bar-Yam. arXiv, March 6, 2012.