Warren Buffett has a message for public pensions, colleges and the like: Stop pouring money into expensive, high-end money managers.

"The commission of the investment sins listed above is not limited to 'the little guy.' Huge institutional investors, viewed as a group, have long underperformed the unsophisticated index-fund investor who simply sits tight for decades," Buffett wrote in his latest letter to Berkshire Hathaway shareholders. Buffett has long been a critic of so-called alternative investing, a category that includes hedge and private equity funds, among others. The reason is the cut they take for their services, which can make billions of dollars for the managers but far less for clients, according to the man sometimes called "The Oracle of Omaha." "A major reason has been fees: Many institutions pay substantial sums to consultants who, in turn, recommend high-fee managers. And that is a fool's game," Buffett wrote on underperformance.

Read MoreKey Fortress fund stumbles, but cash keeps coming

Institutional investors include pension funds, university endowments, foundations and sovereign wealth funds managed on behalf of countries. Buffett already has his money where his mouth is. His famous "Million-Dollar Bet" with hedge fund-focused investment firm Protégé Partners is that a simple S&P 500 index fund managed by Vanguard would beat a mix of five funds of hedge funds over 10 years. Through seven years, Buffett's index fund is up 63.5 percent while the five funds of funds selected by Protégé are up an estimated average of 19.6 percent, according to a Fortune report in February.