As for the rest of us, there's good news and bad news, particularly for those at or near retirement.

Still, 401(k) and IRA millionaires are relatively rare: The number of retirement millionaires represents 1.6% of the 27.2 million IRA and 401(k) accounts managed by Fidelity.

Fidelity, the largest 401(k) provider in the United States, released its quarterly analysis Thursday. The report on retirement trends has become increasingly relevant as baby boomers retire in record numbers.

Thanks to record-breaking markets and more retirement savings, the number of 401(k) and IRA millionaires has reached a new record, according to Fidelity.

Boomers — they are now 56 to 74 years old — have an average balance of $210,400, but it's well-known that small groups of super-savers — the 401(k) millionaires — push the averages up.

Drill down into the results, and the situation is particularly difficult for baby boomers — the huge segment of society born between 1946 and 1964 who are now entering retirement age in record numbers.

The median — where half have more and half have less — is a far-more modest $69,900.

That leaves very little to draw down on a yearly basis. Assuming a 5% yearly drawdown, that's about $3,500 a year.

"As the median amounts in this study show, millions of Americans over the age of 55 have too little saved for a comfortable retirement, and not enough time to save significantly more," David John, senior strategic policy advisor, AARP Public Policy Institute, told CNBC. "They will have Social Security, but not much else. This will continue until every employer offers some form of retirement benefit and every American can save for retirement from the day they go to work until the day they retire."

Of course, that is not the complete retirement picture. There is Social Security and pensions. And some have more than one retirement account. For example, if someone takes a new job but rolls their old 401(k) into an IRA, their new 401(k) would have an initial balance of $0, which would keep the median and average down.

More from Invest in You:

No 401(k)? No problem. Here's how to save for retirement when you're your own boss

Josh Brown says rushing into the market is like eating junk food

5 powerful money hacks from financially savvy people

Fidelity notes that those who have been in their 401(k)s longer have higher averages, which makes sense. Among individuals who have been in their 401(k) plan for 10 years straight, the average balance reached a record of $328,200.

Still, even Fidelity admits more needs to be done. "Millions of people rely on a 401(k), 403(b) or IRA as primary vehicles for retirement savings, so the industry needs to continue to find ways to make these accounts more accessible, more efficient and easier to use," Kevin Barry, president of Workplace Investing at Fidelity Investments, said in a statement.

The recently passed SECURE Act should help: It raises the minimum age for required minimum distributions, which allows older investors to keep more in their retirement plan for longer periods. It allows investors to contribute to traditional IRAs after turning 70.5 years old and makes it easier for administrators to offer annuities, which under some circumstances may be an attractive way to reduce risk.