It looks like the rich have finally shaken off the recession.

The number of U.S. households with a net worth of $1 million or more, excluding primary residence, rose to 9.63 million in 2013, according to a new report from Spectrem Group, a consulting and research firm.

That's more than a 600,000 leap up from 2012, and the highest number on record.

Related: The Super Rich are mad as hell - and doing great

This is the first year that the number has surpassed the pre-recession high of 9.2 million in 2007. Once the global financial meltdown hit and the bottom fell out of the market, the number tanked to 6.7 million in 2008.

"The last few years, we've seen the number continually increase, but this was the first year that we're finally beyond the economic crisis," said George Walper, Jr., president of Spectrem Group.

He said that the market reaching all-time highs last year, coupled with an improving real estate market, drove the rich to get even richer.

Related: The richest (and poorest) in Congress

The ultra-wealthy weren't lagging behind. Walper said both the number of households with a net worth of $5 million and above and $25 million or more reached the highest numbers since Spectrem started tracking the figures.

There were 1.24 million households with a net worth of $5 million or more last year, up from 840,000 in 2008. Those with $25 million and above climbed to 132,000 in 2013, up from 84,000 in 2008.

"Because of their levels of wealth, they increased their exposure to equities during the downturn and were making investments in real estate when the market was at the bottom," he said. "That's why now they've benefited the most from the return in the economy."