Are all millionaires wealthy? Not if you ask them.

A whopping 70% of those with at least $1 million in assets that are invested or available to invest, excluding home values, don't consider themselves to be wealthy, according to a survey of 4,500 affluent investors by UBS. Rather, it's only when they hit the $5 million mark that millionaires begin to feel "wealthy."

Why $5 million? Apparently, that's the level at which most rich people feel they have "no constraints on activities," according to the survey.

Related: Best advice now for getting richer

In addition to feeling like money is no object, most wealthy people also find it important to hold a substantial amount of their fortune in cash because it helps them feel more secure.

"Holding a significant amount of cash is is a critical component of investor confidence, as investors believe these are assets they won't lose," the survey said. "Investors aren't quick to forget the significant losses they endured in 2008."

For the past several years, affluent investors have been holding about 20% of their assets in cash. And though they are aware that cash can lose value over time due to inflation, wealthy investors say they still need significant cash holdings to give them peace of mind.

Calculator: When will you be a millionaire?

Even though they're setting aside a sizable portion of their assets for a rainy day, a majority of respondents are optimistic about the economy and their personal financial situations over the long term.

Most of them aren't too worried about the Federal Reserve's eventual policy changes either, according to the survey. Although recent messages from the central bank and Fed chief Ben Bernanke have sparked volatility in the stock market, most affluent investors believe that the central bank's eventual move to end its stimulus policies will help stabilize the economy.

And a majority of them aren't changing their investment strategy in light of the Fed's upcoming changes.