Osama bin Laden's al-Qaida is under heavy pressure in its strongholds in Pakistan's remote tribal areas and is finding it difficult to attract recruits or carry out spectacular operations in western countries, according to government and independent experts monitoring the organisation.

Speaking to the Guardian in advance of tomorrow's eighth anniversary of the 9/11 attacks, western counter-terrorism officials and specialists in the Muslim world said the organisation faced a crisis that was severely affecting its ability to find, inspire and train willing fighters.

Its activity is increasingly dispersed to "affiliates" or "franchises" in Yemen and North Africa, but the links of local or regional jihadi groups to the centre are tenuous; they enjoy little popular support and successes have been limited.

Lethal strikes by CIA drones – including two this week alone – have combined with the monitoring and disruption of electronic communications, suspicion and low morale to take their toll on al-Qaida's Pakistani "core", in the jargon of western intelligence agencies.

Interrogation documents seen by the Guardian show that European Muslim volunteers faced a chaotic reception, a low level of training, poor conditions and eventual disillusionment after arriving in Waziristan last year.

"Core" al-Qaida is now reduced to a senior leadership of six to eight men, including Bin Laden and his Egyptian deputy, Ayman al-Zawahiri, according to most informed estimates. Several other Egyptians, a Libyan and a Mauritanian occupy the other top positions. In all, there are perhaps 200 operatives who count.

The most significant recent development is evidence that al-Qaida's alliance with the Taliban in Pakistan and Afghanistan is fraying, boosting the prospect of acquiring intelligence that will lead to Bin Laden's capture or death. Despite an intensive US-led manhunt, there has not been a credible lead on the Saudi-born al-Qaida leader in years. Bin Laden's nickname among some CIA hunters is "Elvis" because there have been so many false sightings of him.

"Al-Qaida has become a liability for the Taliban," said Mustafa Alani, a terrorism expert at the Gulf research centre in Dubai who visited Waziristan in July. "There is a good possibility that the Pakistanis or the Americans will be able to get good intelligence on the ground and kill Bin Laden."

Intelligence agencies are watching closely to see if Bin Laden issues a message marking tomorrow's 9/11 anniversary, as he has in the past, or leaves it to Zawahiri. Last week one Islamist website promised a "Ramadan gift" from the al-Qaida leader but removed the posting without explanation.

Amid a mood of cautious optimism, some experts talk of a "tipping point" in the fight against al-Qaida. Others argue that only Bin Laden's death will bring significant change. But most agree that the failure to carry out spectacular mass attacks in the west since the 2005 London bombings has weakened the group's "brand appeal" and power to recruit.

"In order to stay relevant al-Qaida have to prove themselves capable and they haven't been able to do that," said Norwegian scholar Brynjar Lia.

Popular sympathy, which drained away because of sectarian killings in Iraq, has dwindled further this year. In Saudi Arabia, according to a recent intelligence report, 60-70% of information about al-Qaida suspects now comes from relatives, friends and neighbours, not from security agencies or surveillance.

Another weakness is in the so-called "war of ideas". This week imprisoned leaders of the Libyan Islamic Fighting Group began publishing a "revision" of their previous understanding of jihad.

"The text in itself is probably not a landmark work of Islamic jurisprudence, but it is important because it adds to … a corpus of treatises by former militants challenging al-Qaida on theological grounds," Thomas Hegghammer of Harvard University said on the Jihadica website. "Of course, no one text is going to change the world, but put together, these treatises will constrain al-Qaida's recruitment pool somewhat."

Despite a largely positive balance sheet, no one claims the fight against al-Qaida is over or that "victory" can be declared. Robert Mueller, the director of the FBI, said in an interview with USA Today this week: "Yes, they retain the capability of striking overseas. They are still lethal." Intelligence experts say the trends are favourable but point to the IRA maxim that "you only need to get lucky once".

Jonathan Evans, the head of MI5, warned this year that Somalia could become a safe haven for al-Qaida in the way Afghanistan was in 2001. Analysts speak of worries that al-Qaida activity in North Africa and the Sahel could spread to northern Nigeria and affect the UK.

"You haven't lost all those people who were susceptible to the al-Qaida message, ideology or recruitment," said Richard Barrett, co-ordinator of the UN's al-Qaida and Taliban Monitoring Team. "The [al-Qaida leadership] are really under pressure now but could regroup. The conditions remain there, the social factors are all still in place."