Retail stocks have slumped this year.

While many chalk up the declines to competition from online retail behemoth Amazon (AMZN) - Get Report , famed market expert Peter Schiff, CEO of Euro Pacific Capital, doesn't agree.

"Amazon has been here for a long time - this is the worst year for retail - it's worse than 2008 and 2009," he said. "This is because consumers are broke - they have lousy jobs, they're loaded up with debt and they can't afford to buy stuff."

He pointed to Foot Locker's (FL) - Get Report 25% decline last Friday.

Schiff isn't comforted by the double-digit gains in earnings growth for the past two quarters in U.S. stocks.

"Let's see what happens if we go to a recession which is going to impeded [companies'] revenues and let's see what happens if interest rates finally rise despite what the Fed does and that's going to undermine their costs because you have a lot of companies that have levered up their balance sheets."

Schiff said companies have been resorting to debt-financed share buybacks amid low interest rates. "All that's going to come back to bite these companies when interest rates go up," Schiff said.

Watch the full interview with Peter Schiff:

More of What's Trending on TheStreet: