The U.S. public pension gap has tripled to at least $2 trillion in less than a decade, Moody's Investors Service said in a report on Thursday. Read MorePensionsstick with hedge funds despite CalPERS exodus

Moody's measured the unfunded liabilities for the 25 biggest public retirement systems between 2004 and 2012. The total future shortfall is more than half the size of the $3.7 trillion municipal bond market, which comprises all the outstanding debt issued by U.S. states and cities.

People protest in front of the U.S. Courthouse in Detroit during the city's bankruptcy eligibility trial. Getty Images

The gap has widened in spite of the fact that average investment returns over about the same period were 7.45 percent, roughly in line with targets, according to the Wall Street credit rating agency. "Part of the problem for the level of overall funding is that it is inherently difficult to recover an overall asset position after the double-digit losses seen during the recession,'' Al Medioli, a Moody's vice president, said in a statement. Read MoreRough road ahead for'pension smoothing' In fiscal years 2008 and 2009, at the depth of the economic downturn, the plans' assets dropped nearly 22 percent cumulatively on average, Moody's said. Other contributing factors include inadequate contributions from plan sponsors and the burden of an aging population across the country.