Archbishop Alexios of Gaza mopped his brow wearily in the upstairs office of his church, transformed in recent days into a shelter for hundreds of families displaced by the Gaza conflict.

His grey clerical robe undone at the neck and damp with perspiration, the archbishop was exhausted. Last night, the area around the St Porferios Greek Orthodox church – including its little cemetery – was shelled by Israeli forces.

"I only got to sleep at 6am," he said. "We're co-operating with the mosque next door. We're looking after around 1000 people between us." All of the people sleeping in his church are Muslims.

"People started coming in on Sunday, more and more people, so we couldn't even think about holding our Sunday service. Now it's crazy," he added.

The vast majority of those in his church fled the heavy fighting in neighbouring Shujai'iya, which saw scores killed on Saturday and Sunday in the most intense Israeli assault of the conflict so far.

"We gave them blankets and water and money for food," he said. Then on Monday evening, the area around the church itself came under fire.

From the roof terrace outside his office, where two women were sweeping up debris, the damage was clearly visible – the walls of the Protestant school next door are peppered with shrapnel, tombs are broken open or utterly destroyed. A church-owned house next door, where a family who had fled from Beit Lahia in the north had sheltered for a while, is damaged and empty.

"It started after the Iftar meal at about 7.30. The shrapnel even came into the monastery and destroyed our water tanks. The shelling was very strong and the children were crying," the archbishop explained.

"They have an eye up in the heavens," he said, nodding his head upwards toward the ever-present Israeli drones. "I don't believe they intended to attack the church, but we are only a kilometre from Shujai'iya and we are in Zeitoun so the shells have been coming at us from both sides."

A younger priest, Father Amfirohios, was more blunt: "The Israeli military know where we are."

A man entered the Archbishop's office with a pile of shrapnel collected from the church grounds, razor sharp fragments that he held out for inspection.

Inside the stone walls of the church on Tuesday, those sheltering were lying exhausted on benches or sleeping on the tiled floor of the church hall as children played around them.

Mohammad Sukr and his family were among the first to arrive at St Porforios a week ago after the Israelis ordered residents of Shujai'iya to evacuate.

"The Israelis were shelling the open fields at first then they started hitting houses. We were going to go to one of the UN schools at first but then someone told us that the church was open," Sukr explained. He had returned home briefly on Sunday during a brief humanitarian truce and found his home had been damaged by the falling rubble of a neighbour's house.

Sukr was interrupted by the sound of shell fired a little way off, followed by the deafening boom of a rocket launched nearby out of Gaza towards Israel, which sent people running scared into the church's hall.

"It was terrifying last night," he continued. "The shells were falling all around [the church]. We tried to escape and were at the gate when the priest brought us back in."

It is not only the archbishop's church that has been filled by those driven from their homes by the fighting. With more than 100,000 Palestinians displaced – according to UN estimates – UN schools have been opened as shelters. Others people have found refuge with relatives, tens of people crowding into single rooms. Others have occupied shops and offices in Gaza's city centre.

Israel – including its Prime Minister Binyamin Netanyahu – has blamed Hamas for the high rate of civilian deaths, insisting Hamas has turned Gazans into "human shields" or suggesting that Palestinians themselves are somehow responsible for not fleeing when the Israeli army has ordered them to. Gaza's residents have a different explanation for why they have fled – and why they haven't.

The Gaza Strip is a densely populated enclave – around 1.6 million people in 146 square miles, sealed by both an Egyptian and Israeli blockade. Israel has now ordered some 43% of the territory to be evacuated. The schools that have opened to receive those fleeing are already overcrowded – and many Palestinians in Gaza can recall how in the 2008-09 conflict, even those schools could not guarantee safety.

In the first days of the conflict, the Hamas-run ministry of interior issued a statement urging people near the border – who had been told in Israeli leaflet-drops, text messages and phone calls to evacuate – that they should remain in their homes, dismissing the messages as "psychological warfare".

In the past week, the Guardian has seen large numbers of people fleeing different neighbourhoods – including Beit Lahia, Beit Hanoun, Shujai'iya, Zeitoun and Maghazy – and no evidence that Hamas had compelled them to stay..

These crowds have joined the ranks of some 102,000 displaced who, according to the UN's figures, have swollen by some 500% in the last five days.

The decision whether or not to flee is, in reality, a very difficult and complex process of weighing up often competing safety considerations.

"Where you live, you know who your neighbours are and they know you," said one middle class Gaza resident who recently evacuated and asked not to be named.

"You know if someone is Hamas or Islamic jihad, if your building might be a target or if there is someone you suspect is an Israeli collaborator in the building, which might make it safer during an Israeli attack. I'm staying with relatives. The first thing I asked before I moved my family is who the neighbours are."

Others explained their reluctance to flee their homes with their conviction that nowhere in Gaza is currently safe.

Sami Abu Arar has moved with his extended family to a car shop in the centre of Gaza City; a crowd of men were camped in the little office opening onto the street, the women were out of sight upstairs. A hundred metres away, a group of journalists was waiting for a building that had received a warning shot from an Israeli shell to be hit.

Another family occupied the barber's shop next door, women and children crowded into the tiny space.

"I came from Zeitoun [in the south of Gaza City] three days ago with 30 of our family," Abu Arar explained. "My father owns this place. We came here to stay with our relatives. It's still not safe even here. There are strikes everywhere and you don't know if you'll end up living next to a wanted person who the Israelis want to kill."

Asked why he has not gone to one of the UN schools, he responded with a dismissive tutt.

"There's no room and nowhere to sleep – they're all full. But if the tanks come in any closer we'll go to one of the schools."

A few hundred metres away on Omar Mukhtar Street – in the very centre of Gaza – the body of Inas Derbas hung from a section of concertinaed concrete still clinging to the side of a 10-storey tower block – the Salam Building. She had been living on the fifth floor with 101 other people. The bomb that killed her and six members of her family had sheared an entire side of the block's top five stories clear off. Someone, somehow, had managed to reach her body and cover it with two blankets, one purple and one white.

One of her relatives, Mohammed Hussein, stood in the crowd below the block as pieces of concrete continued to tumble down.

"They were two families," he said. "They fled from Beit Lahia five days ago because of the fighting there."

First they had gone to Shujai'iya – the scene of the heaviest bombardment of the conflict so far, resulting in the highest single death toll. "They stayed there until it was too dangerous, then they came here and rented an apartment."

If they had thought they had reached safety – in a place where nowhere is truly safe – they were fatally mistaken.