With last year’s presidential election so focused on jobs and the economy, the American public probably knows more about the nuances of the unemployment rate than they ever have before. One particular of the official unemployment rate that received attention last year is that the Labor Department counts someone as unemployed only if he or she is actively looking for employment. Other folks who, for whatever reason, aren’t searching for work are considered not in the labor force.

The ratio between those considered in the labor force and the total working-age population is known as the “labor-force-participation rate.” And the reason why the unemployment rate has been able to fall from more than 10% in 2009 to 7.6% today despite middling job growth is that more and more Americans are dropping out of the labor force altogether.

(MORE: The Female Labor Market Is Actually Stagnating)

As you can see from the chart below, this decline in the labor-force-participation rate is a trend that’s been going on for many years now. The primary driver of the overall trend is the aging workforce — many of those dropping out are simply retiring at around the normal age. But the trend accelerated during the recession, suggesting that many more people dropped out of the workforce than otherwise would have if the economy were in better shape.

US Labor Force Participation Rate data by YCharts

As the economy improves, however, should we expect to see the participation rate bounce back? According to new analysis from Macroeconomic Advisers, it’s not likely. They estimate that roughly 45% of the recent decline in labor-force participation is a result of a weak economy, and the rest because of demographic factors. But as the economy improves, the workforce is going to continue to age, meaning that by 2015, when the Federal Reserve expects the economy to be back near full employment, the participation rate will remain where it is today.

In one sense this is good news because the economy doesn’t need to produce as many jobs per month to see reductions in the unemployment rate. But on a deeper level, it’s evidence that the high labor-participation rates that helped spur economic growth from the 1970s through the 1990s is a thing of the past. To put it another way, our economy is going to have to produce more with fewer people going forward, which — all else being equal — will slow economic growth.

(MORE: Forget ‘Buy American’? U.S. Retailers Push an ‘Imports Work’ Campaign)

So just how big of an effect will a smaller workforce have on the economy? In 2011, Harvard University’s Program on the Global Demography of Aging published a paper that tried to understand this question. Logic suggests that a workforce that has to support fewer nonworking individuals will be wealthier overall, and the study bears out that hypothesis. In the paper, economists David E. Bloom, David Canning and Günther Fink estimated how high-income countries (most of which have aging populations) would have grown from the 1960 to 2005 period if they had experienced population growth similar to the projections for 2005 to 2050. According to the report, if a high-income country like the U.S. had a per-person income of $10,000 in 1960, that income grew to $34,600 under the population growth we actually experienced. On the other hand, if the 1960 to 2005 period experienced the sort of population growth that we’re expected to see through 2050, that income would have grown only to $25,500.

This huge difference underscores how important population growth and workforce participation is to a country’s economy. If there are more people working in a country, and if a higher percentage of those people are productive, the whole country will be richer. (This is one reason why economists tend to support policies that increase immigration.)

There are bright spots, however. First, as the economy moves away from manufacturing and toward less-physically-demanding services, workers will be able to work later in life, easing the downward pressure on the participation rate. Second, one should not lose sight of the fact that increased longevity is a good thing in and of itself. Economists care about economic growth only because it helps increase human welfare, but a longer life can help in that endeavor as well.

That being said, society will have to support these longer lives somehow, and part of the price we pay will most likely be slower economic growth.

MORE: Job Satisfaction or Long-term Job Stability? Turns Out It’s Hard to Have Both