It's time to reduce the amount of physical cash floating around the globe to help combat tax evasion and other illicit activities, Harvard professor and renowned economist Ken Rogoff told CNBC on Tuesday.



"I'd eventually phase out the $100 note. I'd phase out the $50. And over 10 or 20 years, I'm phasing out the $20s," he argued, while explaining the thesis of his new book "The Curse of Cash" in a "Squawk Box" interview.

"Cash is very good way to port, horde, hide for crime [and] tax evasion. The evidence, I think, is overwhelming that that is a lot of the use," he added. "There are things like illegal immigration … that's cash-driven. If you couldn't pay in cash, it wouldn't drive all the illegal immigration."

But society needs to find a sweet spot on cash, so there's enough for regular consumers, who can buy things, but not enough for criminals to exploit, Rogoff said. "Leaving some around makes sense."

"There are always going to be ... gold coins, uncut diamonds and now bitcoin. As long as you can't go spend them at the store; take them to the bank," he said. "You have to play whack-a-mole with all these things."

While bitcoin makes a lot of headlines, the digital currency remains "very low scale now," he said.

"If you look at cash outstanding in the world, there's tons of it. It's mostly in large denomination notes, $100 bills. There are more than 4,000 for every man, woman and child in the U.S. We print more constantly," Rogoff said. He estimates nearly 80 percent of all cash in circulation is in $100 bills.

Producing all this cash is a profit-making endeavor, Rogoff said. "The government makes a fortune off printing cash." But he said, "It's penny wise and pound foolish" when considering the relatively small amount of revenue generated versus the money lost to tax dodgers.