The ultra-rich were the largest losers in dollar terms. Households in North America with investible assets of more than $100 million saw their wealth decline 2.4 percent. Their population declined slightly to 2,928 from 2,989.

The main reason for all this wealth loss? Stocks.

With the wealthy today increasingly dependent on stocks for wealth, last year’s stalled stock market shrunk the population of millionaires and nicked the fortunes of existing millionaires. According to BCG, the amount of wealth held in equities declined 3.6 percent last year.

Globally, the picture looked a little brighter. Virtually all of the growth in global millionaires came from emerging markets last year. While the United States lost nearly 130,000 millionaires, the rest of the world added 175,000 millionaires. There are now 12.6 million millionaire households globally, according to BCG.

The country with the highest “millionaire density” – proportion of population who are millionaires – was Singapore. More than 17 percent of Singapore’s households are millionaires.

While 2011 saw declines in U.S. millionaires, the future looks brighter, BCG said. They expect wealth held by millionaires globally to grow at around six percent a year in Asia-Pacific. “We are in a two-speed world,” said the report. “All of the growth is being driven by developing markets.”

-By CNBC's Robert Frank

Follow Robert Frank on Twitter: @robtfrank



