Americans in their 20s and 30s are facing a retirement crisis that could plunge them back into the Great Depression, Blackstone President and Chief Operating Officer Tony James said Wednesday.

"Social Security alone cannot provide enough for these people to retain their standard of living in retirement, and if we don't do something, we're going to have tens of millions of poor people and poverty rates not seen since the Great Depression," he told CNBC's "Squawk Box."

The solution is to help young people save more by mandating savings through a Guaranteed Retirement Account system, he said. Right now, young people cannot save enough on their own because they face stagnant incomes and heavy student-debt burdens.

Read More Tug-of-war: Fund retirement or kid's college education?



The Guaranteed Retirement Account was proposed by labor economist Teresa Ghilarducci in 2007 as a solution to the problem of retirement shortfalls that inevitably arise when contributions are voluntary.



A GSA system would require workers to make recurring retirement contributions, which would be deducted from paychecks. Employers would be mandated to match the contribution, and the federal government would administer the plan through the Social Security Administration.

Ghilarducci has proposed a mandatory 5 percent contribution, but James said a 3 percent requirement rolled into GRAs could outperform retirement savings vehicles like IRAs and 401(k)s.



Read More New grads may have to work until they're 75



He noted that a 401(k) typically earns 3 to 4 percent, while a pension plan yields 7 to 8 percent. The average American pension plan has a 25 percent allocation to alternative investments — including real estate, private equity and hedge funds — with the remainder invested in markets, he said.

"The trick is to have these accounts invested like pension plans, so the money compounds over decades at 7 to 8 percent, not at 3 to 4," he said.

A 25-year-old who earns 3 to 4 percent per year would retire with $75,000, not nearly enough to annuitize and live on, James said. A 7-percent-per-year investment would yield $200,000 at retirement, he said.



Read MoreWhich 10 states are the least prepared for retirement?



Under the plan James is proposing, the government would offer a 2 percent guarantee on GRAs.

"The key to it is taking that capital, setting up the Guaranteed Retirement Accounts and investing it well for the very long term," he said. "We have to do that and we have to do that professionally."

James spoke ahead of the Center for American Progress's conference on creating more inclusive prosperity and promoting long-term planning in the private sector.