If you try to trade in and out of the market, "your emotions will defeat you totally," Bogle added . "Short-term betting is not a good way to go."

In the long run, there is a high correlation between the stock market and U.S. economic growth, he said, so, regardless the economic climate, "Stay the course. Don't let these changes in the market, even the big one [like the financial crisis] … change your mind and never, never, never be in or out of the market. Always be in at a certain level."

"If you hold the stock market, you will grow with America," Bogle, founder of The Vanguard Group, said on CNBC's " Power Lunch ."

Legendary investors Warren Buffett, 88, and John C. "Jack" Bogle, 89, agree on the key to successful investing: Buy and hold the stock market for the long term.

Buffett, who runs Berkshire Hathaway and has accumulated an $89 billion fortune over his career, has always maintained a long term view when deciding whether or not to invest in a company.

"Nobody buys a farm based on whether they think it's going to rain next year," he said on CNBC's "Squawk Box." "They buy it because they think it's a good investment over 10 or 20 years." In other words, the Oracle of Omaha looks for companies that will last for decades, not just ones that are doing well right now.

"Put together a portfolio of companies whose aggregate earnings march upward over the years, and so also will the portfolio's market value," Buffett wrote in his 1996 letter to shareholders. "If you aren't willing to own a stock for ten years, don't even think about owning it for ten minutes."

Even when the market seems to be tanking, keep a level head and stay the course, he says. In response to wild market fluctuations back in 2016, Buffett told CNBC that buy-and-hold is still the best strategy.

"Don't watch the market closely," he advised those worried about their retirement savings at the time. "If they're trying to buy and sell stocks, and worry when they go down a little bit … and think they should maybe sell them when they go up, they're not going to have very good results."

Don't miss: Jack Bogle reveals the biggest risk of the next financial crisis and why he's still investing

Like this story? Subscribe to CNBC Make It on YouTube!