Viktor Orban’s comments come as refugees are forced off train headed towards Austria and taken to refugee camp.

Hungarian Prime Minister Viktor Orban has said that his country does not want to take in large numbers of Muslims, in defence of Hungary’s response to the surge in refugees trying to enter the country.

“I think we have a right to decide that we do not want a large number of Muslim people in our country,” Orban told journalists outside the EU headquarters at Brussels.

“We do not like the consequences,” he said, referring to the country’s 150-year history of Ottoman rule during the 16th and 17th centuries.

Orban said those fleeing conflict in countries such as Syria should not try to cross into Hungary, as he defended the country’s decision to erect a fence along its border.

“Please don’t come… It’s risky to come. We can’t guarantee that you will be accepted,” Orban said in Brussels, adding that it would not be humane or morally right to “falsify” people’s dreams.

Refugees detained on Austria-Hungary border

“We Hungarians are full of fear, people in Europe are full of fear because they see that the European leaders, among them the prime ministers, are not able to control the situation,” Orban said.

His comments came as refugees who boarded a train bound for the Austrian border clashed with Hungarian police as they were forcefully unloaded and taken to a refugee camp.

Clinging to railway tracks

Thousands of refugees had been sleeping rough outside the Budapest train station as police blocked them from entering for two days.

In Pictures: A night of protest at the train station

After they were eventually let in on Thursday morning, a packed train departed bound for Sopron, a town near the Austrian border.

But Hungarian police stopped the train before its destination, with police forcing refugees off and directing them onto buses to take them to a refugee camp, state news agency MTI reported.

Some families clung to railway tracks after trying to run away from police.

About 50 riot police were lined up as a replacement train allowed non-refugee passengers to continue their journey, Reuters reported.

Orban’s chief of staff said police would stick to the EU’s Schengen rules and make all checks needed on refugees travelling on domestic trains towards the country’s western border.

Hungary has been widely criticised for its way of handling the flow of refugees to Europe.

A fence on the border with Serbia is one of several measures under way to make it more difficult for refugees to enter and stay in Hungary. The government is also tightening asylum laws, introducing penalties for illegal border-crossing, and planning to close permanent refugee camps.

Why Al Jazeera will not say Mediterranean ‘migrants’

EU President Donald Tusk, meanwhile, warned that divisions between western member states and their newer eastern partners were complicating efforts to solve the deepening refugee crisis.

“There is a divide … between the east and the west of the EU. Some member states are thinking about containing the wave of migration, symbolised by the Hungarian [border] fence,” Tusk said.