When last we checked in on Jeju Island, the little South Korean resort that unexpected found itself dealing with hundreds of refugees from the brutal civil war in Yemen, the refugees were cautiously optimistic about the help they were receiving from private citizens and the South Korean government.

Three weeks later their situation has reportedly taken a turn for the worse, with homeless refugees camping on the streets of Jeju.

Korea Joongang Daily cited a report from the Jeju Immigration Office on Friday that around 40 of the refugees have been found sleeping in public places since June 8. On that same day, seven more of them were found sleeping in a public park. Another group of refugees was found setting up a tent city on a beach.

“We’ve had some Yemenis come to us saying they are about to go broke and have no choice but to sleep in the open,” said an immigration official.

The Yemenis have been barred from traveling to mainland Korea, so the provincial government issued a special exemption from the normal labor laws allowing them to seek employment with Jeju’s fishing and hospitality industries. According to Korea Joongang Daily, local employers have “complained about cultural gaps” with the new arrivals:

A fish farm owner told the JoongAng Ilbo that he was confused when a Yemeni worker he hired suddenly stopped working in the middle of a day to pray. Another asylum seeker said he quit his job at a pork restaurant because the owner would only serve his workers pork, which is forbidden in Islam. “The owner had no understanding that we Muslims don’t eat pork,” said the asylum seeker, who would only allow the paper to refer to him as Fahro.

Jeju remains a fascinating microcosm of the worldwide refugee crisis. The tiny island lacks the resources to deal with hundreds of Yemenis, who took advantage of a quirk in South Korean visa policies to relocate to Jeju after temporary asylum in Malaysia expired. The islanders want legal protection for their culture and tourist industry. Many are already exasperated by the special exemptions hastily imposed for the Yemenis, and as the latest updates indicate, those measures provided only temporary relief for the refugees. Mainland South Korea has granted asylum to only a very small fraction of the applicants since it embraced the U.N. Refugee Convention in 2013.

For their part, the refugees are out of money, have nowhere else to go, and strongly wish to avoid returning to their own country.

The total number of Yemenis involved is quite small, less than 500 in total, but Jeju has a small permanent population. Mainland South Koreans are worried that granting special concessions to the Yemenis could prompt a larger flood of refugees.

They are also displeased with measures that bend or ignore existing laws instead of passing new legislation, including growing public exasperation with the refusal of President Moon Jae-in’s Blue House to answer a petition concerning the refugees even though it long ago passed the required number of signatures to compel an official response. The petition calls for tightening South Korea’s visa policies and processing asylum applications more quickly. Until now, asylum-seekers have been allowed to remain in the country while their processing grinds along for months or years.

South Koreans who express opposition to asylum for the refugees have been accused of selfishness, racism, and Islamophobia. Protesters sometimes refer to the Yemenis as “fake refugees,” pointing out that they had enough money to purchase plane tickets to Malaysia and then Jeju, and that a high percentage of the asylum-seekers are unaccompanied young males.

Nikkei Asian Review noted last week that some of the Yemenis look more like “economic migrants” than desperate people fleeing a war-torn country with death on their heels, profiling one 23-year-old asylum-seeker who lived in Malaysia for a year on “money sent from home” before deciding Jeju was a better destination because he “heard South Korea was taking in refugees.”

The article pointed out that most of the Yemeni refugees are men “because they are much more likely to be sent to the battlefield,” and since they are “generally not fleeing poverty and are decently dressed,” they do not fit the popular image of terrified and desperate people with nowhere left to run.

Some South Koreans feel that since their country has no presence in the Middle East and no colonial history there, they should not be required to accept refugees from Middle Eastern conflicts. President Moon is himself the son of refugees from North Korea, but South Koreans view such refugees very differently from those who originate elsewhere.

The South Korean Ministry of Justice announced on Monday that seven of the refugees were allowed to leave Jeju and come to the mainland for medical treatment or to be reunited with relatives.

A recent survey found South Koreans opposed to granting asylum to the Yemeni refugees by a margin of 53-37, with opposition growing steadily across most regions and demographics.