A major Islamist terror attack in Europe is almost inevitable as European members of Islamic State (Isis) return from Syria and Iraq, according to senior EU officials familiar with the diplomatic, intelligence, and security planning taking place to try to counter the threat.

They said the EU’s bodies and its 28 governments were under intense US pressure to get to grips with the menace represented by thousands of European citizens fighting in Syria, but that Europe was struggling to develop coherent instruments to reduce the risk of an atrocity.

“It is pre-programmed,” said a senior official involved in the policy and security debate over the chances of an attack. “We have clear signals that this is what the foreign fighters are doing. This is the main threat we are facing.” Interior ministers from the 28 countries are to meet in Luxembourg in a fortnight to try to come up with a concerted policy.

“The home affairs council is very aware and very frightened of this … The colleagues in the police administration just don’t know how to cope. They all fear this could be totally out of control. It may already be too late,” the senior official told the Guardian and five other European newspapers.

In a separate interview, Gilles de Kerchove, the Belgian EU official who coordinates the union’s counter-terrorism policy, said executives from the big social media providers, including Twitter, Facebook and Google, would attend the interior ministers’ meeting in Luxembourg in an EU attempt to deprive Isis propagandists of their highly effective exploitation of the internet.

“We want these companies to develop a counter-narrative. There will be a big discussion with the internet players,” said De Kerchove.

He put the number of EU citizens fighting in Syria at around 3,000. “We don’t have harmonised statistics. But the flow of fighters has not dried up. It’s a significant number and it has not stopped,” he said.

Senior US intelligence and homeland security officials have been attending recent meetings of EU policy-makers, alarmed that some of the European fighters could be easily infiltrated into the US.

“The Americans are very worried about Europeans entering freely under the visa waiver programme. They are looking into this very seriously,” said De Kerchove.

In addition to the dilemmas posed by extremists returning to Europe, EU capitals and Washington are aghast not just at the brutal prowess shown by Isis in Syria and Iraq, but also at the claimed arrival in Syria of senior al-Qaida operatives from havens in Afghanistan and Pakistan grouped in the so-called Khorasan group. They are said to include the Saudi explosives expert Ibrahim al-Asiri.

“This guy seems to be one of the best bomb-makers in the world,” said De Kerchove. “This small group of veterans linked to al-Qaida is a concern. We know some flew from Afghanistan/Pakistan to Syria.”

Confronted with these dilemmas, EU interior ministry, intelligence and police officials are meeting regularly in various combinations. But the attempts to come up with a coherent policy and instruments are dogged by institutional, national and departmental rivalries and differing priorities, senior officials said.

The EU has been trying to come up with a counter-terrorism strategy for the past 18 months. The current emergency is jolting the process, but officials are intensely pessimistic that the results will be too little, too late.

Various schemes are under discussion, most notably an EU-wide Passenger Names Record (PNR) for all air travel within the EU supplying up to 15 parameters that are mixed in a computer algorithm to help identify suspects.

The scheme is opposed in the European parliament on civil liberties grounds as it would monitor millions of ordinary travellers. The Germans, sticklers for data protection, are also lukewarm on the idea but are keener on reintroducing tighter border controls within the passport-free Schengen zone.

“We think PNR is one of the few tools allowing detection of suspicious travellers,” said De Kerchove. “But many people think it’s a dangerous slippery slope, collection of data on the innocent.”

At recent EU meetings with his counterparts in the EU, Thomas de Maiziere, the German interior minister, has also been urging more rigorous screening of all passports and ID cards at airports, the officials said. The proposal was also opposed on the grounds that it would cause massive queues.

A police database known as SIS or Schengen Information System is also available as a tool for flagging up suspicious travellers and identities that have been entered into the system. The intelligence services, the sources said, are wary, however, of contributing information to this system for fear of compromising their material, thus rendering it less effective.

Britain, the source of around one quarter of the European jihadis believed to be in Syria, is not party to the SIS system because it opted out of all the instruments under the EU’s justice and home affairs portfolio and still has to negotiate what bits it will rejoin.

But according to De Kerchove, the British are nonetheless among the most active and insistent in pushing a tough concerted EU strategy.

“We should have the UK plugged into the SIS. That’s very important, but it has not happened yet. The Home Office says they want to be in. I was in London last week. They push and push. On counter-terrorism, the UK is one of the countries supporting us the most. They’re very, very committed and they have excellent information.”

A Foreign Office spokeswoman said the UK authorities believed “more than 500 UK-linked individuals have now travelled to Syria and Iraq since the uprising began. Obviously, it’s very difficult to give precise numbers on this.”

De Kerchove said he had asked Theresa May, the home secretary, and had not received precise figures. Nor was it clear how many had returned from Syria or Iraq to Britain.

According to the French authorities, the number of native jihadis in Syria and Iraq has soared from 555 to 932 this year. Of those, 118 have returned to France. According to experts consulted by European officials involved in the effort, an estimated one in nine of those returning represents a terrorist threat.

Officials point to the killing of four people at Brussels’s Jewish Museum in May as a portent of things to come and of the mishaps afflicting the Schengen system. The suspect awaiting trial in Belgium, Mehdi Nemmouche, 29, a French national of Algerian heritage, had spent a year in Syria. He flew from Turkey to Frankfurt in Germany. German customs officers identified him from the SIS system and alerted the French. He was allowed to travel further and allegedly came to Brussels where he is suspected of opening automatic machine-gun fire on the museum before being arrested later in Marseille.

The number of EU nationals fighting in Syria is put at 3-4,000. The senior official said that in post-9/11 Afghanistan there were an estimated 100 Europeans fighting with al-Qaida and the Taliban and that presented a big problem then.

Two Dutch nationals of Turkish origin were also arrested by the Belgians last month on their return from the Middle East, with Dutch television reporting at the weekend that they were plotting an attack on the headquarters of the European commission in Brussels. The officials said there was no evidence to support this.

On Thursday the mayor of Brussels, Yvan Mayeur, said the threat of returning jihadis “is not virtual for us, it is concrete, it is real”.

He said he was examining 14 files on the issue of suspected extremists from Belgium, which is believed to have the highest per capita rate in the EU of fighters in Syria.