Defense Secretary Jim Mattis recently criticized Venezuela’s socialist President Nicolas Maduro as a “despot” who will ultimately “have to go” but indicated the U.S. would not use military force to make it happen.

Mattis spoke at the Reagan Defense Forum over the weekend and was asked by moderator Fox News’s Bret Baier if he foresaw the U.S. military having to do something in Venezuela where a refugee situation is worsening and the country is spiraling out of control.

“I think what we should do is work by, with and through our Latin-American allies, and we stabilize the situation. Right now, you will see we have a U.S. naval ship, Comfort, a hospital ship, it’s down there providing medical aid to many of these refugees,” he said on Saturday.

“We are doing our best diplomatically and through humanitarian aid, but ultimately this regime is going to have to go, and it’s up to the Venezuelan people, it’s up to the regional states in that area to help expedite that, and bring that country back to a more prosperous and positive future,” Mattis added.

More than three million Venezuelans have fled the oil-rich country in recent years due to food and medicine shortages, hyperinflation, and violent crime, according to Reuters.

The USNS Comfort has treated “thousands” of Venezuelans, according to the Associated Press.

The Trump administration has imposed sanctions on Venezuela and denounced Maduro for trampling on human rights and worsening the economic situation. Maduro had blamed the country’s problems on U.S. sanctions and his political adversaries.

Mattis said despite the worsening situation in Venezuela, democracy is spreading throughout the rest of the Western Hemisphere.

“Democracy is actually spreading. We have three countries, Cuba, Nicaragua, Venezuela, who are living in the past. They’re not going the right direction,” he said.

“The rest of the hemisphere, from Canada to the tip of Chile and Argentina, is going to the right direction. Those countries are very concerned down in that region with what’s going on in Venezuela. It is tragic. Again, it’s what happens when you have an irresponsible despot leading a country to ruin.”