Thinking about bidding Uncle Sam goodbye? Not so fast. Renouncing U.S. citizenship isn't cheap, and can result in enormous tax bills for decades to come.

A record 3,000 Americans chose to give up their passports last year, driven in part by new regulations that have turned tax preparation into a massive headache for the 7.6 million U.S. expats required to file returns while living abroad.

But as more Americans are finding, ditching the U.S. can be an expensive affair. Earlier this year, the government increased the renunciation fee to $2,350, more than four times what it used to cost.

On top of that, some U.S. citizens are slapped with a giant "exit tax" bill -- sometimes millions of dollars -- when they renounce.

The increased renunciation fee alone "is frustrating enough for the little old lady school teacher down the street," said Chris McLemore, senior counsel at Butler Snow. "But if there's somebody in exit tax land, it's obviously a much more expensive process."

The tax pain can last for years, with some former Americans on the hook for additional payments decades after they renounce.

Related: 3,000 Americans ditch their passports

"This is what really boggles everybody -- you're going to go into the U.S. embassy, you're going to swear an oath of renunciation, you're going to hand over your passport, and yet in some cases, you'll never be free from a tax perspective," said McLemore. "At the same time, you've given up all your legal rights as a citizen."

Top income earners, or those with $2 million or more in global assets -- salary, homes, retirement savings, artwork, jewelry and stocks all count -- pay tax on the current market value of their total holdings, as if they had just been sold.

"[You're] going to pay immediate tax, which effectively, is the price for the way out," McLemore said. Few wealthy individuals are exempt, and paying a professional to figure out the bill can cost thousands.

Related: Why expats are ditching their passports

Take for instance Facebook co-founder Eduardo Saverin, who was criticized for renouncing and moving to Singapore prior to Facebook's massive 2012 IPO. Saverin was on the hook for tax on gains built into his shares of the company, and any other assets he may have owned, said Summer LePree, a lawyer with Bilzin Sumberg.

"I am obligated to, and will pay hundreds of millions of dollars in taxes to the U.S. government," Saverin said in a statement at the time. "I have paid and will continue to pay any taxes due on everything I earned while a U.S. citizen."

No investment is safe: An American can be taxed on the full value of pension and retirement plans, even if those funds won't be paid out for years, said Dianne Mehany of Caplin & Drysdale.

In certain cases, assets that were subject to the exit tax could again be taxed after the owner's death -- if they are inherited by U.S. citizens, for example.

For many Americans, the decision to renounce is a gamble that could end up costing far more than staying, said McLemore.

"You're triggering a tax now, without actually taking the proceeds," he said. "This is why people with actual exit tax exposure are having to think long and hard on whether they just keep the blue passport."

Read next: Nobody escapes U.S. taxes - even astronauts