Despite recent gains on their investments, U.S. public pension funds don't have nearly enough money to pay what they owe current and future retirees.

In less than a decade, that shortfall has tripled to at least $2 trillion—more than half of all outstanding state and local bond debt, according to a report by Moody's Investors Service.

Moody's looked at the unfunded liabilities of the 25 biggest public retirement systems, which cover 40 percent of the $5.3 trillion in total U.S. public pension plan assets.

Like nearly all retirement savers, pension funds got clobbered by the 2008 financial collapse. In the 2008 and 2009 fiscal years, the 25 plans' assets dropped nearly 22 percent cumulatively on average, Moody's said.

But the pension shortfall had been building well before the downturn—and has been made worse by state and local government's shortchanging annual fund contributions. New Jersey, for example, took "contribution holidays" during the Great Recession and more recently has cut payments or just skipped them altogether, Moody's said.

States and cities have also used accounting gimmicks to mask the widening shortfall, including "asset-smoothing" that lets them spread out the impact of the market downturn. They've also used rosy scenarios to inflate their estimated investment returns. Unlike corporate accounting rules, the rules for government accounting let pension fund managers just pick a return number that makes their future liabilities look smaller.

Read MorePensions stick with hedge funds despite CalPERS exodus



"As a result, a common practice has been to use a relatively high discount rate that mathematically understates plan liabilities … relative to other approaches, such as those supported by financial economists," Moody's said in its report.