The surging stock market helped boost average 401(k) balances to yet another record high in 2013. But many young and low-income workers are not doing such a great job of keeping that cash in their accounts.

The average 401(k) balance hit $89,300 at the end of the year, up 15.5% from $77,300 in 2012, according to an annual tally by Fidelity Investments. Most of the boost came from stock market gains as all three major stock indexes ended the year more than 20% higher.

People on the verge of retirement, ages 55 to 64 years old, saw their nest eggs grow to an average balance of $165,200 from $143,300 in 2012, Fidelity said. Savers with both a 401(k) plan and Individual Retirement Account managed by Fidelity had larger nest eggs, with an average balance of $261,400, up from $225,600 in 2012.

Related: Will I have enough to retire?

Even a balance of $261,000 is hardly enough for a comfortable retirement. And many Americans are much more woefully unprepared for retirement. A 2013 study by the Employee Benefit Research Institute found that nearly half of workers had less than $10,000 saved.

Part of the problem: Many workers are putting their retirement savings at risk when switching or leaving jobs by not rolling over the accounts to 401(k)s or IRAs.

Of the roughly 800,000 workers with Fidelity accounts who left a job in the first nine months of 2013, 35% cashed out their 401(k) balances, as opposed to leaving the money in their former employer's plan or rolling it into a new 401(k) or IRA. While Fidelity said the statistic was "concerning," it was not a significant increase from previous years.

Related: Which one first: student loan or 401(k)?

These cash-out balances averaged $15,500, and were especially common among young and low-income workers. More than 40% of participants between the ages of 20 and 39 and 50% of workers earning between $20,000 and $30,000 had opted for the cash.

Unless workers use the funds to open an IRA account within 60 days, they get hit with significant taxes and a 10% penalty.

It also makes it harder for their savings to grow. For example, a 30-year-old who cashes out $16,000 could lose nearly $500 in monthly retirement income, assuming she retires at 67 and lives to the age of 93, Fidelity said. Plus, the 30-year-old would be hit with $3,200 in taxes and another $1,600 in penalties at the time she cashes out.

Related: How to be a 401(k) millionaire

"Everyone's personal financial situation is different and there are times when a person must have access to cash," James MacDonald, president of Workplace Investing at Fidelity, said in a statement. "However, we urge all investors -- especially young savers with years of potential investment gains -- to keep their 401(k) savings working for them."

An earlier version of this article incorrectly stated how long 401(k) participants who cash out have to open an IRA without getting hit with taxes and penalties. They have 60 days.