Despite recent market swings, some retirement savers are feeling pretty comfortable these days.

The number of Fidelity 401(k) plans with a balance of $1 million or more jumped to a record 196,000 as of June 30, up 17% from 168,000 last year, according to a new report by Fidelity, the nation's largest provider of 401(k) plans. The financial services firm handles more than 30 million retirement accounts altogether.

In addition, the number of individual retirement account millionaires increased to 179,700, also an all-time high.

More from Invest in You:

Trick yourself into better money management. No budget required

6 ways to break the fear cycle and get started in investing

Invest in what you know and keep it simple

Average retirement savings balances are now over six figures, according to Fidelity. When broken down by generation, millennials, or those at the early stages of their retirement planning, had an average balance of $29,000, while baby boomers averaged $198,000.

Overall, the average 401(k) balance is $106,000, just shy of the all-time high balance of $106,500 reached in the third quarter of 2018, while the average IRA balance is $110,400 as of the second quarter, also slightly below last year's high of $111,000, Fidelity found.

Still, nearly a third of savers increased their contribution rate during the last year, Fidelity found. The average 401(k) contribution rate is now 8.8% — not including the employer match — the highest percentage ever.

On top of that, employers are contributing another 4.7%, on average, bringing the total contribution to 13.5%. "That's really close to the 15% we believe people should be saving," said Katie Taylor, a vice president at Fidelity.

As a rule of thumb, Taylor recommends saving 10 times your income by retirement age. Although $1 million is "a threshold that a lot of people anchor on," Taylor said, a more accurate picture of your retirement number should be based on how much you make and your lifestyle.