A large number of people have missed the six-year bull market.

More than half of Americans, 52 percent, are currently not investing in the stock market—either by buying individual stocks or mutual funds, or through a retirement account such as a 401(k) or IRA, according to a new Bankrate.com survey. Among the non-investors surveyed, 53 percent said they don't have the money to invest and 21 percent said they don't trust stock brokers or financial advisors.

"It's a definite negative, especially for younger people who can afford to take some risk right now because they have time to recover from potential losses," said Claes Bell, a Bankrate.com analyst. "For most people, stocks should be some part of the mix in their overall retirement portfolio because of that ability to earn higher returns over a long period of time."



Read MoreRetirement savings last on most Americans' to-do lists



What's worse is that Bankrate's survey of more than 1,000 adults may be overestimating the percentage of Americans investing in the stock market. A study from the Federal Reserve found that only 13.8 percent of U.S. households owned stocks in 2013, down from 15.1 percent in 2010.