By Madeline Chambers

BERLIN (Reuters) - The head of the European Parliament implored citizens to fight the "demons" of racism and anti-Semitism that still haunt Europe in a speech to mark the liberation of Germany's Buchenwald concentration camp 70 years ago.

Buchenwald, near the city of Weimar, was the biggest concentration camp on German soil. Set up by Hitler's SS in 1937, it held more than 250,000 Jews, Roma, homosexuals and other people not tolerated by the Nazis. More than 56,000 people died there from torture, medical experiments and starvation.

Some 80 camp survivors traveled to Weimar from Europe, the United States and Israel, joining U.S. army veterans who helped free Buchenwald on April 11, 1945. On Saturday they held a minute's silence at 3.15 pm, the time of liberation, and laid red carnations at the mustering point.

Fears are growing about anti-Semitism in Europe, underscored by the January attacks in France on Charlie Hebdo magazine and a Jewish supermarket, at a time when immigration has shot to the top of Germany's political agenda.

European Parliament President Martin Schulz said in a speech in Weimar that in the 1930s people allowed the seeds of hatred to grow in their hearts, allowing the Holocaust to happen.

"To honor the victims...We want to fight the return of demons that we thought were overcome but which still show their ugly face - racism, anti-Semitism, ultra-nationalism and intolerance," said Schulz.

He homed in on events in the eastern town of Troeglitz, where neo-Nazis are suspected of setting fire to a refugee home and whose mayor resigned because he feared an attack from right-wing radicals over his plans to house asylum seekers.

Although towns across Germany have seen similar attacks by right-wing radicals - Der Spiegel says there were 150 last year - the fire in Troeglitz caught the headlines.

"We must not let agitators and arsonists believe a silent majority stands behind them," said Schulz. "We must oppose xenophobia and say clearly that we, the majority, support refugees," he said.

The debate on immigration, caused partly by a 60 percent surge in asylum seekers to Germany last year, has been fueled by the grass-roots anti-Islam PEGIDA (Patriotic Europeans Against the Islamisation of the West) group in Dresden.

Although Germany has also seen big counter-demonstrations and its popularity has waned, Dutch far-right politician Geert Wilders is to address marchers on Monday and may boost numbers.

(Reporting by Madeline Chambers; editing by Susan Thomas)