Billionaire hedge fund manager Leon Cooperman defended his industry, saying passive management isn't how famed investors have built their fortunes.

"All I know is if the ability to underperform exists, the ability to outperform also exists. Warren Buffett, Mario Gabelli, Stan Druckenmiller and Ken Langone — and a little bit Lee Cooperman — didn't get to their net worth by buying an index," Cooperman said Wednesday on CNBC's "Halftime Report."

Passive investing has grown in popularity as hedge funds have seen billions in outflows. Unlike hedge funds or active management, passive funds track an index and don't have professional managers selecting their holdings.

While some market watchers may be right that hedge funds are having a tough time in the short term, Cooperman argued history shows that may be the wrong call in the long run.

There are periods where the hedge fund industry consistently outperforms the market, but also stretches where it struggles to do that. When hedge funds can't deliver, he said it makes sense that their asset bases shrink.

"Money goes where money is treated best. The one truism is that you cannot charge a premium fee and deliver subpar performance," Cooperman said.

He said a bear market or "two-way market" could help hedge funds regain their popularity.

"There's no question it's an expensive form of asset management and fees are coming down," Cooperman said.

Cooperman's Omega Advisors had about $3.6 billion in assets under management as of March 31.

Watch: Cooperman positive on MSFT