Last year was a banner one for Americans giving up their passports.

The number of U.S. citizens who revoked their status reached an all-time high of 3,415 people, a jump of 14 percent from 2013, according to deVere Group, a financial advisory firm.

Avoiding Uncle Sam appeared to be a prime motivator, after a new U.S. tax law was enacted that makes it harder to hide assets from authorities. A survey from deVere last year found that almost four out of five of its clients, who are primarily American expatriates, said they would consider handing in their passports because of the law, which is called the Foreign Account Tax Compliance Act, or FACTA.

Get Breaking News Delivered to Your Inbox

"Treasury Department data show that a steadily growing number of individuals have been giving up their U.S. citizenship over the last few years," said deVere Group founder Nigel Green in a statement. "It can be reasonably assumed that this trend is in direct response to complying with the onerous, expensive and privacy-infringing FATCA, which finally came into effect on July 1 last year."

While the numbers picked up sharply last year, the trend for Americans handing in their passports has been rising since 2010, the group noted, when FACTA was originally enacted by Congress. The law, which went into full effect last year, requires foreign financial institutions to report the financial holdings of U.S. clients, or else face a 30 percent withholding tax on a range of payments from the U.S.

The five highest annual totals of Americans giving up their passports have been recorded since 2010, when the law was enacted, the deVere Group noted. Even though the numbers have been rising, it's still a small fraction of the 7.6 million Americans who reside outside U.S. borders.

While the law was targeted to track down tax evaders, critics say it has ended up hurting middle-class Americans who live outside the country and infringes on their privacy.

Still, for many the decision to give up their citizenship is a last resort, deVere Group noted. "For most Americans the very idea of giving up their citizenship because of the hassle and expense of complying with a controversial new tax law is unpalatable, to say the least," Green said.

Aside from FACTA, America's tax laws on expatriates have caused controversy and an outcry over what some say is unfair taxation. The U.S. is the only country that taxes its citizens on worldwide income, instead of where they earn the money. That means that an American who lives in Australia, for instance, has to file taxes in both countries, although they are exempt from paying U.S. taxes on their first $97,600 of earnings.

The surge in passport-swapping Americans may slow as citizens grow comfortable with the new regulations, the deVere Group noted.

It's not a step to take lightly: once an American gives up their passport, the move is not reversible. The U.S. State Department cautions those considering the move to think it through carefully, calling the step "serious and irrevocable."