Despite a surge of compassionate support for the plight of refugees sparked by photos of a drowned Syrian toddler face down on a Turkish beach, pockets of violence against those seeking sanctuary in Europe remain, with attacks against asylum seekers in both Greece and Germany.

On the Greek island of Kos, a group of 15 to 25 men attacked refugees with bats and yelling “go back to your country,” according to Amnesty International staff, who witnessed the violence first hand. Police had to disperse the crowd using tear gas, according to the rights groups.

Also on Friday, five people were injured in a fire at a refugee shelter in Heppenheim, a small town in western Germany. The three-story building housed migrants from countries such as Somalia, Syria, Iraq and Ethiopia. In recent months, there have been more than 100 suspected arson attacks against refugee shelters.

German Chancellor Angela Merkel has condemned the spate of arson attacks, which came as resistance against immigration and resettlement of refugees in some part of the EU has increased.