The rich are getting richer, but not in every country.

The number of ultra wealthy people in the world ticked up by 0.6% in 2015, according to a new report from Wealth-X, while their total wealth increased by 0.8% to $30 trillion -- a record high.

Together, the super rich make up just 0.004% of the adult population, but they control 12% of the world's wealth.

Wealth-X defined ultra high net worth individuals as those with at least $30 million in net assets.

While the number of ultra rich people and their wealth climbed to record highs last year, not everyone was raking in the dough.

Depressed energy and commodity prices made life a little more difficult for the wealthy in more energy dependent countries.

Related: 4 reasons OPEC will struggle to boost oil prices

Russia, Norway and Australia all suffered hefty declines in the population and wealth of their richest citizens.

Oversupply in the oil market has pushed down prices, which have been hovering below $50 a barrel for most of the year.

Australia experienced the biggest decline in its ultra rich population of any developed economy, according to Wealth-X.

The country is a big exporter of natural gas, coal and iron ore. And one if its main trading partners, China, is experiencing an economic slowdown. That's bad news for Australia, since roughly one third of its export market heads to China.

Related: The gender gap keeps widening, even among billionaires

The number of ultra wealthy people in Australia slipped almost 31% to 2,475 last year with a total wealth of $295 billion, a 33% decline.

Russia also saw its wealthy population slip, dropping nearly 13% to 1,075 individuals who have $30 million or more in assets. In its capital city of Moscow, the number dropped 30%.

"If oil stays [this low] long term, that could have much larger consequences," said Benjamin Kinnard, research analyst at Wealth-X. "And some of these people impacted will have to make long-term changes to adapt rather than ride it out."

Norway's rich also took a hit. The Scandinavian country, which is a big oil exporter, saw its ultra wealthy population drop almost 54% to hold $90 billion in wealth -- a 51% decline from 2014.

Related: Blame oil: Nigeria slumps into recession, Norway stalls

While the global growth in wealth and number of ultra high net worth individuals was flat last year, the richest of the rich still had a good year. Billionaires' wealth rose by 5.4%.

The wealthy in countries that import energy or have manufacturing-focused economies also did well.

The United States is home to the most ultra wealthy people at 69,000 or a third of the world's total super rich.

Female wealth also took a hit last year around the world: their share of total ultra high net worth wealth dropped to 11% from 14%.