Billionaire Warren Buffett told CNBC on Tuesday that it's a "terrible mistake" to eliminate the estate tax as part of the Republican approach to tax reform.

"It's not a death tax," the chairman and CEO of Berkshire Hathaway said on "Squawk Box."

"There are going to be 2.6 million people [who] die this year in the United States," but there will be only about 5,000 tax returns that will owe estate taxes, he said in a wide-ranging interview.

Buffett said because the estate tax impacts only a small percentage of Americans, getting rid of it would not cause widespread problems.

"If they pass the bill they're talking about, I could leave $75 billion to a bunch of children and grandchildren and great-grandchildren. And if I left it to 35 of them, they'd each have a couple billion dollars," Buffett said. He then asked rhetorically, "Is that a great way to allocate resources in the United States?"

Using a sports metaphor to argue against that kind of dynasty building, Buffett said, "I don't think we should have our 'Olympic team' 20 years from now be the eldest sons of the 'Olympic team' currently."

"The wealthy now are so much wealthier than they were 25 years ago," he said. "We're talking about the 400 [richest] now having $2.4 trillion ... 25 times as much money."

Buffett accounted for $75.6 billion of it as of Monday's stock market closing prices. He's second on the Forbes billionaires list behind Microsoft co-founder Bill Gates' $86 billion fortune.

The so-called Oracle of Omaha, who rarely makes business decisions based on Washington, told CNBC earlier Tuesday that Berkshire is holding off on selling to see how tax reform plays out.

"I would feel kind of silly if I realized $1 billion worth of gains and paid $350 million in tax on it if I just waited a few months and would have paid $250 million," Buffett said.

Republicans last week unveiled their blueprint for tax reform, calling for cutting personal and corporate tax rates and aiming to simplify the U.S. tax code.