It fell to a middle-aged Roman Catholic nun to take down the controversial black Islamist flag that had flown for several days in the shadow of Canary Wharf.

Sister Christine Frost, a well-known community activist in east London, was yesterday morning preparing to take a group of residents on a day trip to the seaside at Clacton-on-Sea, when she and members of the local Muslim community decided the banner fluttering provocatively at the entrance to the Will Crooks estate had to be removed.

It was similar to the flags flown by jihadi fighters and had appeared in the largely Muslim area following a Gaza solidarity protest the previous week. But Sister Christine reasoned that it was just too much like the flags flown by the Isis group which has beheaded non-believers on its mission to create a caliphate in Iraq and Syria. It risked destabilising community cohesion, she thought.

But as the banner disappeared from the Poplar housing estate on Friday, a toxic dispute over its meaning began.

The flag had intimidated Christian residents, a city banker admitted to being unnerved by its presence, and on Thursday night there were accusations of anti-semitic behaviour by a group of youths hanging around underneath it.

Many other residents in the area – which has a large Bangladeshi Muslim population – said the black flag was simply a emblem of solidarity with the people of Gaza and represented cross border unity between Muslims.

The Arabic writing on the flag, they said, was simply the creed, or shahada, of Islam: "There is no god but Allah and Muhammad is his messenger." But it was nevertheless inescapably similar to the "flag of jihad", often used as a backdrop by suicide bombers and Islamist militants over the past 25 years.

So had the standard of the most violent, intolerant and terrifying interpretation of Islam been raised for the first time in the capital, just a half a mile from the skyscrapers of the city's principal financial district?

Whatever the reading of the flag's meaning, Tower Hamlets council, which oversees one of Britain's most multi-cultural boroughs and a population which includes a third from Bangladesh, was sufficiently concerned to say that "there is no place for hate in Tower Hamlets, whatever form it takes".

On Friday, the morning after the presence of the flag was first published by the Guardian, council officials took steps to remove the emblem only to find that it had already been taken down by Sister Christine and her helpers.

"They had no idea of the implications of the black flag being taken over by Isis fighting in Iraq, in the same way as the BNP appropriated the union jack and gave a bad feel to it," Frost later told the East London Advertiser.

It had been fixed to the apex of a metal archway alongside two banners demanding an end to the siege of Gaza and a Palestinian flag.

One Christian resident of the area, who asked not to be named, told the Guardian that he found the flag "very intimidating given what Isis are doing in Iraq to Christians and others". Anyway, he said, "an entry to a housing estate shouldn't have the creed of Islam hanging on it".

A white banker who worked for HSBC at its nearby Canary Wharf headquarters considering buying a flat in the area, saw the flag and worriedly asked a local Muslim cafe owner if it was a sign he would be bullied by the largely Muslim community if he moved in.

"As soon as we realised it was being misrepresented we took it down," said Ruhel Choudray, 31, a community worker at a local mosque. "No one thought of that black flag as a jihadi flag or Isis. They just wanted to support Gaza. There has been a big misunderstanding and the community is really shocked."

Mubin Ahmed, 36, a manager at a care support agency on the estate, also said the black flag did not denote jihadi sympathies and was an emblem of unity in Islam across borders in the face of the Israel's attacks on Gaza.

"There is excessive force by Israel and not a single country in the world is doing anything about it," Ahmed said. "I think it is another holocaust. I get the feeling that they are inflicting what the German Jewish community experienced."

Rabia Begum, 31, a cafe owner on the estate, laughed at suggestions it was related to Isis and said the "boys" who she suspected put it up would not have fully understood its connotations. It was more to do with local people's desire to make a statement about what is happening in Gaza. "When you realise people are living a big prison, don't have anything or the means to get anything, you think we have a voice and if we can make a comment it is a good thing," Begum said. Others, however, argued the flag could not have been flown without there being a more provocative intention. "The black flag with the Muslim declarations of faith written in white is known as the rayah, a symbol that has been adopted by Islamists in the last 25 years," said a spokesman for the anti-extremism thinktank, Quilliam.

They added: "Since the Bosnian conflict, it has been increasingly adopted by jihadist organisations. It is a provocative move by those who have put up the flag as they will well know its significance."

• This article was amended on 9 August 2014. It originally referred to city bankers being unnerved by the flag's presence. It was in fact one city banker. This has been corrected.