The numbers seem to tell a pretty clear story. The typical American family is poorer than it was 25 years ago.

Median family wealth has dropped to $81,450, a nearly 5% decline since 1989, according to data compiled by the Federal Reserve Bank of St. Louis.

That doesn't seem so surprising since income has stagnated over the past quarter century and the middle class is struggling amid rising costs.

But a closer examination of the numbers tell a more nuanced tale.

Looking at how each race has fared since 1989, you'll find that each one has gotten wealthier.

Asians, whites, blacks and Hispanics are seeing their net worth go up for several reasons. Typically, wealth increases with age and schooling -- and each of these groups is getting older and more educated, said William Emmons, senior economic adviser at the St. Louis Fed's Center for Household Financial Stability. The share of each race's population that is under age 40 has fallen since 1989, while the share of those completing high school has grown.

Also, many Americans have become more responsible for saving for their own retirement, said Caroline Ratcliffe, senior fellow at the Urban Institute. Corporate and government employers have been shedding traditional pension plans and offering 401(k) plans instead. Unlike pensions, 401(k) accounts are considered wealth.

So why is the nation's overall wealth slipping?

Because our population has become much more diverse. Blacks and Hispanics make up a larger share of the population than they did in 1989. And while blacks and Hispanics are wealthier than they were 25 years ago, they still have accumulated far less wealth than whites. So as their share of the population increases, the national median net worth dips.

The black population in the U.S. grew by 30% over the past 20 years to 38.9 million in 2010, according to Census data, while Hispanics increased by 126% to 50.5 million. The white population is also growing, but much more slowly, up only 4.6% to 196.8 million. The Census data shows the Asian population as growing 173% to 22.5 million, but the category also includes other smaller racial groups, as well as biracial Americans.

The wealth gap between these groups stems from large differences in earnings and homeownership. Blacks and Hispanics are less likely to own homes, which is a major source of Americans' wealth, and they earn less, making it harder to build a nest egg. And while people of color have increased their retirement savings, the value of their savings lags badly behind whites.

Whites are also five times more likely to receive large gifts or inheritances, according to an Urban Institute report published earlier this year.

The Great Recession also hit blacks and Hispanics harder than it did whites. The net worth of whites depends more on the fortunes of the stock market, which has come roaring back. But blacks and Hispanics' wealth is more tied to their homes, and housing prices have recovered more slowly.