It's wrong for people to think that they have to be wealthy to get rich investing in the stock market, famed buy-and-hold investor Ron Baron told CNBC in a Friday interview.

"You have to have a small amount of money and invest it regularly for a long time, and live to get to be old. That's how you get rich," said the billionaire founder of Baron Capital, which has nearly $26 billion in assets under management. He appeared on "Squawk Box" from the sidelines of his annual investor conference in New York City.

"It's all about compounding," he said, referring to the power of making regular investments and reinvesting the returns over decades.

Baron has made billions of dollars by doing extensive research, buying the stocks of what he feels are undervalued companies, and keeping them for an average of about 14 years. But individual investors don't need to pick stocks, he said. "The simplest thing for people to do is buy an index fund."

"If you invest $5,000 a year for 30 years ... it's worth $890,000" based on historical stock market returns, he calculated.