With the U.S. economy in full recovery mode, the share of Americans in the workforce should be on the rise, right? Not quite.

Thirty-five percent of the U.S. population wasn't in the labor force in 2014, compared with 31.3 percent just 10 years earlier, according to a recent study from the Bureau of Labor Statistics. Americans who don't have a job and aren't searching for one are distinct from those counted by the unemployment rate, which measures those without jobs but who are looking to get hired.

The rise in workforce dropouts may be surprising on one level, given that the recession is long over and employers continue to add workers. Yet in 2014, 87.4 million Americans over the age of 16 weren't in the labor market, compared with 70.5 million in 2004, the BLS said.

Demographics partly explain the jump because baby boomers started hitting retirement age in 2011.

Get Breaking News Delivered to Your Inbox

But it's not only boomers who say they're kicking back into retirement. The number of Americans reporting they've entered their golden years increased in every age group. Take Americans who are between 20 to 24 years old: In 2004, only 48,000 said they were retired. By 2014, that had increased to 132,000.

Of course, it could be that those represent the children of the ultra-rich, or young Americans who made fortunes in business, entertainment or other industries. And given their age, it's possible many of them will exit retirement to look for employment in the near future.

Another culprit behind the shrinking workforce participation rate may be a more serious issue: the rise of illness and disability. About 16.3 million Americans said they couldn't work because of health issues in 2014, up from 12.4 million a decade ago.

While even younger Americans increasingly claim health problems for sidelining them, the biggest bulge is among men and women between 55 to 64 years old, the BLS said. About one out of eight Americans in that group said they aren't able to work because of illness or disability, compared with slightly more than one out of 10 in 2004.

Those issues of illness and early retirement may go hand-in-hand, research from Boston College's Center for Retirement Research have found. Americans who suffer from poor health are "significantly more likely to retire early than others," the research center said in a study published last year.

What's the impact for those who remain in the workforce? If the trend continues, it could become tougher for employers to find workers, which could help boost wages, Princeton University economist and former Federal Reserve vice chair Alan Blinder told Bloomberg News.

The BLS report has another glimmer of good news: More Americans cited educational pursuits as a reason for dropping out of the workforce. About 6.4 percent of all people over the age of 16 are enrolled in school, up from 5 percent a decade ago.

Given that educational achievement is linked with higher incomes, that trend could signify rising earnings down the road for the millions of additional people who are enrolling in college or graduate programs.