Puerto Rico, whose finances were already in a shambles, has been struggling to recover from Hurricane Maria, which struck the island two weeks ago as a Category 4 storm. Most of the 3.4 million residents are still without electricity and water.

Trump visited the hurricane-devastated U.S. territory on Tuesday. On Tuesday night, he told Fox News that Puerto Rico's debt will have to be wiped out. "They owe a lot of money to your friends on Wall Street and we're going to have to wipe that out. You're going to say goodbye to that, I don't know if it's Goldman Sachs , but whoever it is you can wave goodbye to that," Trump said.

The island's general obligation bonds, which yield 8 percent, dropped to just 37 cents on the dollar. Just last month, the bonds were trading at around 56 cents to the dollar.

Puerto Rican bonds took a huge hit on Wednesday following President Donald Trump's comments on the island's massive debt.

President Donald Trump and first lady Melania Trump walk through a neighborhood damaged by Hurricane Maria in Guaynabo, Puerto Rico, U.S., October 3, 2017.

"This is not something a president should be doing or can do," said Larry McDonald, head of the U.S. macro strategies at ACG Analytics. "It's just noise, and it's pretty far removed from reality."

"This is not a dictatorship. We have bankruptcy judges and the rule of law," McDonald said. "But it is scaring the bond market."

Municipal bond insurers with exposure to Puerto Rico's debt also saw their stocks fall. Shares of MBIA, Ambac Financial and Assured Guaranty declined 8.4 percent, 5.5 percent and 2.9 percent, respectively.

Mick Mulvaney, director of the Office of Management and Budget, tried to walk back Trump's comment on Wednesday. In an interview, he told CNN: "I wouldn't take it word for word with that. I talked to the president at length yesterday as we flew home on Air Force One."

"The primary focus of the federal effort is to make sure the island is safe and that we're rebuilding the island," he said.

Puerto Rico currently holds more than $70 billion in debt in part because of years of government overspending. Cate Long, founder of Puerto Rico Clearinghouse, told CNBC in an email that in aggregate, about 75 percent of it is held by retail investors.

Senate Minority Leader Chuck Schumer told reporters that government needs to find a "fair solution" to solve Puerto Rico's debt problems. "Maybe the horrible, horrible devastation that's been caused to Puerto Rico will make our colleagues understand that we have to have a far fairer solution to Puerto Rico's problems," he said.

Last year, Congress passed the PROMESA law that created Financial Oversight and Management Board. Its goal is to help Puerto Rico negotiate with its creditors.

PROMESA also created a mechanism known as Title III that allowed U.S. territories such as Puerto Rico to enter a form of bankruptcy, a financial tool the island did not have access to previously.

"The president can't just wipe out the debt," said Jeff Lipton, managing director, head of municipal research and strategy at Oppenheimer. "He doesn't have the power to wipe it out. To my knowledge, he cannot issue an executive order even though Puerto Rico is a United States territory."

However, Long of Puerto Rico Clearinghouse said Trump's comments could help debt-restructuring proceedings move faster. "So far it has been mired in court litigation and needs to be moved to a consensual negotiation which was required by law (Promesa), but the PR Oversight Board skipped over," Long said.

Puerto Rico has also been contending with a shrinking population in recent years. Between 2010 and 2016, the island's population has shrunk by 8.4 percent, according to the U.S. Census Bureau. The mass exodus from Puerto Rico could accelerate after Hurricane Maria hit the island two weeks ago.

—CNBC's Patti Domm and Reuters contributed to this report.