Just as single-income families began to vanish in the last century, many of America’s elderly are now forgoing retirement for the same reason: They don’t have enough money. Rickety social safety nets, inadequate retirement savings plans and sky-high health-care costs are all conspiring to make the concept of leaving the workforce something to be more feared than desired.

For the first time in 57 years, the participation rate in the labor force of retirement-age workers has cracked the 20% mark, according to a new report from money manager United Income.

As of February, the ranks of people age 65 or older who are working or seeking paid work doubled from a low of 10% back in early 1985. The biggest spike in employment has gone to college-educated older workers; the share of all employees age 65 or older with at least an undergraduate degree is now 53%, up from 25% in 1985.

This rise of college-educated older workers has pushed the demographic’s inflation-adjusted income up to an average of $78,000, 63% higher than the $48,000 older folks brought home in 1985. By comparison, American workers below the age of 65 saw their average income rise by only 38% over the same period, to an average of $55,000. United Income’s calculations draw on recently released data from the Census Bureau and the Bureau of Labor Statistics.

(More: Congress, states poised to upend retirement policy)​

There’s a mismatch between older workers who need the income the most and those who are able to work and working, said Elizabeth Kelly, senior vice president of operations for United Income and a former special assistant to the president at the White House National Economic Council during the Obama administration.

“These are the more educated, wealthier individuals in better health who are continuing to work, but it’s probably their less-educated, working-class counterparts who need to work the most,” Ms. Kelly said.

The BLS expects the big wave of aging baby boomers to represent the strongest growth in the labor force participation rate through at least 2024.

“By 2024, baby boomers will have reached ages 60 to 78,” a BLS report noted. “And some of them are expected to continue working even after they qualify for Social Security benefits.”

The retirement math is ugly, even for those who are seemingly well-off. Teresa Ghilarducci, an economics professor at the New School for Social Research, has estimated that Social Security replaces about 40% to 50% of one’s pre-retirement income. The general thinking is that people need around 80% of pre-retirement income to get by after they stop working. (Online retirement calculators can give a rough sense for what you need to save, and earn on savings, to get there.)

(More: Richard Thaler wants to use 401(k)s to boost Social Security payments)​

The typical worker in the bottom 50% of the income distribution, earning less than $40,000 a year, has no retirement savings. Those in the middle 40% of income distribution, earning from $40,000 to $115,000, have a median amount of $60,000 saved, according to Ms. Ghilarducci’s research. Workers in the top 10% of income distribution making more than $115,000, meanwhile, have a median amount of $200,000 saved. They, too, are woefully under-saved, although it’s worth noting that these calculations don’t include real estate and other tangible assets, or the chance of an inheritance.

Ms. Ghilarducci’s rough estimate of what a typical college-educated professional must amass to retire fairly comfortably? “Over $1 million or 2.” No wonder more people are working longer.