While most corporate employers shifted from pension plans to 401(k)s after the latter’s creation in the early 1980s, governments still offer pensions to many employees.

And those pensions have left retirees and taxpayers in a bind that has fueled political battles while continuing to en rich investment consultants and managers: Too-optimistic estimates of market return, which determine how much governments must pay to fund the balance, have left many plans massively underfunded even as the advisers who managed them received huge fees.

What’s left? A disaster waiting to happen.

The money to pay future retirement benefits to government workers such as firefighters, policemen, and teachers comes from two sources: contributions made by governments to the funds — that is, by taxpayers — and investment growth. The more the funds’ investments grow, the less taxpayers must contribute.

To ensure that there will be enough money to pay retirees’ benefits later, money must be contributed now. How much depends on future investment returns.

Thus, the assumed future rate of return on investment is very important. The higher the assumption, the less taxpayers must contribute.

Most developed countries, including the U.K., Canada, the Netherlands, Sweden, and France assume a conservative return, as on government bonds. Their pension plans are very well-funded.

In the U.S., the assumption is not at all conservative. U.S. funds assume an average return of 7.6%. If they were to assume the return on a low-risk U.S. government bond, it would be only 2.5% or 3%.

That is why some experts who analyzed recommendations made by the Pension Task Force of the Actuarial Standards Board believe U.S. state and local government pensions could be underfunded by $5 trillion.

Put another way, if those pension funds tried to buy annuities from insurance companies to fund the future benefits they have promised, they would be short $5 trillion.

That’s why the Pension Task Force recommended that pension funds switch to using a “market rate of return”— which would be, for guaranteed future benefits, the much lower return on an annuity or government bond.

Political resistance to making this change is very strong. It would mean that governments — i.e. taxpayers — would have to contribute up to three times as much as they do now.

Consequently, the funds continue assuming an unrealistically high return. They hire consultants and money managers who happily acquiesce, tacitly implying they can achieve it. Trouble is, they can’t.

But both consultants and investment managers — particularly the hedge-fund managers that consultants recommend — charge absurdly high fees. These fees come out of pension funds’ coffers and, ultimately, from taxpayers, unless they default on the promises to retirees.

Consultants have assumed high enough returns for “alternative” investments — that is, hedge funds and private equity funds — to push pension funds to the maximum acceptable allocation to these funds. Completing a vicious cycle, they justify the decision to buy those funds because of the need to meet the aggressive assumptions.

But those funds have performed abysmally. Meanwhile, those managers have siphoned off as much as 60% of their gains to fees. That is why there are so many megarich hedge-fund managers.

It has been known for a while that hedge funds and private-equity funds have performed terribly for their investors, and that their fees are too high. Yet pension funds continue to add to their investments in these funds.

The reasons are plainly political. Lowering the assumed rate of return would result in an immediate requirement for taxpayers to as much as triple their contributions to these pension funds — or for the promised benefits to be curtailed. (That’s illegal, according to some state constitutions.)

Neither Republican nor Democratic politicians want either of these things to happen, though perhaps for different reasons. So they “kick the can down the road.”

The return assumption remains high. The consultants and investment managers continue, against all evidence, to allow their pension fund clients to believe — or to pretend to believe — they can achieve these returns.

Where’s this headed? If, in the future, many government pension funds fail all at once to have sufficient funds to pay their retirees, the burden will suddenly fall on state governments — and ultimately on the federal government.

Meanwhile, hedge fund and private-equity managers and consultants will be happily rich from the fees. It’s a sobering reminder that the needs of retirees, who are the ultimate investors, and the desires of investment advisers and managers aren’t always aligned — and what can happen when they aren’t.

You’re invited: MarketWatch is hosting a free panel discussion on the future of robot — and human — financial advice on April 5 in downtown Boston. RSVP required, continuing education credit available. Learn more or email marketwatchevent@wsj.com to RSVP.

Economist and mathematician Michael Edesess is chief investment strategist of Compendium Finance, adviser to mobile financial planning software company Plynty, and a research associate of the Edhec-Risk Institute. E-mail him at michael.edesess@gmail.com. Follow him on Twitter at @MichaelEdesess.