Lesbos, Greece - A girl, no older than five, dressed in a pink coat sits on the ground behind a white barbed wire fence in the port city of Mytilene, the capital of the Greek island of Lesbos.

Exhausted children, women and men are sleeping on the ground under foil blankets around her.

These are the new arrivals to Lesbos, who have come at a moment of crisis on the island.

Over the weekend, President Recep Tayyip Erdogan said that Turkey would no longer police its border with Europe. In the wake of his announcement on Saturday, there were 518 arrivals on Sunday to Lesbos and 94 on Monday, according to local news reports.

Local residents, who were already angered by the mounting numbers of refugees and migrants, erected road-blocks over the weekend to prevent more being transferred to the infamous Moria camp. Families are now being housed in makeshift accommodation in Mytilene port.

Erdogan's move has come at a critical time as tensions mount on Lesbos.

Just last week, locals clashed with riot police dispatched from Athens during protests over plans for a new migrant detention centre in Lesbos's north.

The mood is increasingly sour on the island, which has over the years shown itself capable of extraordinary solidarity.

In 2016, islanders from the Aegean were nominated for a Nobel peace prize for their efforts to help refugees arriving on their shores in 2015.

Locals have protested against the government's plan to construct detention centres on Lesbos [Julian Busch/Al Jazeera]

But lately, vigilantism has been rising. In early February, Lesbos police arrested seven people on suspicion of planning attacks on refugees and migrants.

There have also been protests by both refugees and migrants decrying living conditions and Greek residents who fear the island cannot cope with more people arriving.

In recent days, following the Turkish announcement, NGO vehicles, journalists and aid workers have been assaulted, some violently, by a small group of local agitators.

Apostolos Veizis, a doctor with Doctors Without Borders (Medicins sans Frontiers, MSF), said while those responsible for the violence "do not represent the island or Greece itself", he was deeply concerned. MSF workers have also been attacked.

"We have never faced such a situation with such violent attacks, and where the government and competent authorities are informed about it and nothing is changing," he told Al Jazeera.

"That I have to discuss this in March 2020 about how solidarity is being targeted in such a mob way, I never thought I would be doing this. I feel like I'm speaking to you from a war zone."

More than four miles away from Mytilene, the overflowing Moria camp continues to stretch outside its borders.

Originally designed to hold no more than 2,840 people, there are now nearly 20,000 people in and around the camp.

Set on a hillside overlooking the Aegean, the panorama presents a sea of tents, noise, music, laughter and the sounds of children playing.

Ahmad, a 33-year-old Afghan refugee, said: "These are the worst days in the history of Lesbos ... when I was in Lesbos almost two weeks ago, for me it was very difficult because of the danger of attacks."

He added that alongside the deteriorating conditions for those in the camp, news of the detention centre in the north was troubling.

"Lesbos is already a prison, Moria is already a prison. People cannot leave the island, and they are going to make a prison inside the prison. This is a sick mind," he said.

At Moria camp, a man builds his tent among a population of nearly 20,000 [Julian Busch/Al Jazeera]

Hanne Beirens, director of the Brussels-based Migration Policy Institute Europe, said EU leaders should be trying to guide public opinion in this volatile situation.

"If leaders can show that we're able to deal with 10,000 people in an orderly way, rather than panicking, you can actually say, 'look, we're in charge here,'."

While it appears that empathy from some communities had worn thin, the situation could change, she told Al Jazeera.

"With different kinds of leadership [and a more] pragmatic approach as to what's happening at the border ... you could actually slowly restore people's confidence in this and I think that's also what's needed."

But Lesbos residents have little faith in European officials, some of whom threw their support behind Greece on Tuesday and issued a warning to Turkey over the two countries' current spat.

At a cafe in Mytilene, George (not his real name), a local raised on Lesbos, told Al Jazeera that vigilante groups did not represent the local community. But, he added, some residents had valid concerns.

"At first, the population was massively helping and assisting the first migration flows and more. However, a few years later, as time went by, it became obvious that this is a permanent situation."

While some businesses in Mytilene may have seen an uptick in custom, tourism profits have dropped for other companies in the north.

"I have never seen Lesbos like this in my experience, I am very saddened," said George.

"Many of the buildings you see here were built by merchants who operated from the Black Sea, down to the Middle East and North Africa. These people were international in perspective, they were never regional or secluded.

"Now we see misery and we feel there's nothing much we can do about it."