"You can't build a museum on the bones of our grandfathers and call it the Museum of Tolerance," Mustafa Abu Zahra told me, as we walked round what remains of the largest Arab cemetery in West Jerusalem. Beyond the trees and the stone tombs that cover the southern half of the cemetery, we could see the white metal fence that enclosed the construction site of a project that has aroused fierce opposition in the six years since its inception. Even its name seems a mockery of the spirit of religious co-operation that the city of Jerusalem - so central to the adherents of three related faiths - is supposed to represent: "It's not about tolerance or love between nations,' said Abu Zahra. 'It's about the violation of a sacred site."

I'd met Abu Zahra at his shop in the Musrara quarter. When I arrived, customers were drinking coffee in the front, and Abu Zahra was receiving visitors at a desk in a storeroom piled high with sacks of rice and tinned goods. Yet he is not just a shop owner and businessman: he is also mutawalli, or guardian, of Mamilla Cemetery. When his guests had left, he drove me round the walls of the Old City to his diminishing realm, amid the air-conditioned shopping malls and upmarket hotels of West Jerusalem.

The journey took only five minutes, but it exposed some of the cultural contrasts that inform the debate about Mamilla's future. Abu Zahra's shop sells everything from figs and spices to cornflakes and cleaning fluid, but once we'd left the market stalls and crowded streets around Damascus Gate, we found ourselves in a very different part of the city.

Mamilla used to lie on the edge of the impoverished no-man's-land that divided the Israeli and Jordanian sections of the city, but since Israel conquered and annexed East Jerusalem in the Six Day War of 1967, it has become a prime piece of land. Jerusalem's best-known hotel, the King David, is 200 metres up the hill; the Waldorf Astoria group is investing $100m (£60m) in another luxury hotel on the street that runs along its southern border. The American consulate in West Jerusalem and Mamilla Mall lie within sight of its gates.

It is probably not surprising that Mamilla's paved avenues, dusty paths and open spaces have gradually been eroded. In 1958, ten years after the state of Israel came into being, its western half was appropriated for Independence Park, and in 1964 a multi-storey car park was built on its northern edge. Yet it is the plan to build the Museum of Tolerance where the car park used to stand that has piqued those such as Abu Zahra, who sees it as nothing less than an attempt to erase the history of the Arab presence in Jerusalem.

The Museum of Tolerance is being developed by the Simon Wiesenthal Centre (SWC), an "international Jewish human rights organisation", named after the renowned Austrian Nazi-hunter. It already owns the Museum of Tolerance in Los Angeles and the Tolerance Centre in New York; in 2004, it inaugurated the Museum of Tolerance in Jerusalem at a ceremony attended by Arnold Schwarzenegger, governor of California and the son of an Austrian policeman who joined the Nazi Party in 1938. The celebrity architect Frank Gehry designed a flamboyant building in steel and glass, but the initial phases of ground-breaking and construction unearthed several hundred skeletons.

Religious and civic organisations demanded that the SWC abandon work and seek another site. The waqf, or religious trust, which is responsible for Mamilla, petitioned the high court to stop the building work, as did a human rights organisation representing three Jerusalem families whose ancestors are buried in the cemetery. In February 2006 the court issued an injunction, and work stopped for two years. But on 28 October 2008 the high court ruled that it could resume, and placed the onus on the Muslim authorities to accept the SWC's offers to reinter the remains elsewhere, clean up the modern Muslim cemetery to the south of the site and establish an appropriate monument to those who were buried there.

Rabbi Marvin Hier, the dean and founder of the SWC, claims that "all citizens of Israel" - Jews and non-Jews - would be the "real beneficiaries" of the decision. "Moderation and tolerance have prevailed. The museum . . . will be a great landmark promoting principles of mutual respect and social responsibility," he says. Others point out that the SWC campaigned for 15 years to remove a Carmelite convent from the grounds of Auschwitz, arguing that nothing should be built on the "single largest unmarked human graveyard in history", and say Mamilla should be accorded similar respect.

Rabbi Hier says the comparison is "ludicrous", not least because "the Arabs" did not treat the site as a cemetery when it was a car park. He maintains that the religious leaders of the Muslim community have ruled that the site was mundras, or abandoned, and says that in 1946 there were plans to build a university on the land. But critics say he has misread the nature of such schemes. According to Yehoshua Ben-Arieh, a professor of geography at the Hebrew University of Jerusalem who is one of the leading authorities on the city's recent history, they were "curiosities" that were never likely to be implemented.

Shattered stone

Besides, what the Mufti of Jerusalem might once have sanctioned is not the point, Ben-Arieh says: what matters is the way in which Israel is treating an important Muslim site located within its sovereign territory. Gershon Baskin, an Israeli Jew who runs a joint Palestinian-Israeli public policy organisation called Ipcri, recalls the reaction when Israel captured East Jerusalem in 1967 and discovered that many graves in the cemetery on the Mount of Olives had been damaged or destroyed.

“Imagine the outrage if the Palestinians were building a Museum of Tolerance - or anything else - on what was once a Jewish cemetery," he says. "Would it matter if the cemetery was not active and in use since 1948, or that it was being done legally?

“The Wiesenthal Centre project in Jerusa­lem is a disgrace to the Jewish people, the state of Israel and the city of Jerusalem. Shimon Wiesenthal would be turning in his grave if he knew what is happening in his name."

When construction at Mamilla resumed, several months after the high court verdict, hundreds more skeletons were exhumed and transferred to a mass grave. It wasn't possible to see what was going on behind the high white fence that sealed the perimeter of the site, but the British artist Sarah Beddington filmed from the windows of a nearby building for a video installation that featured in an exhibition called "The Other Shadow of the City".

Abu Zahra estimates that Mamilla is now a tenth of its original size, and the erosion of its borders is still going on. Recently, a section in the south-eastern corner of the cemetery, beyond the deep stone basin called Mamilla Pool, which was often used as a water source for armies besieging the Old City of Jerusalem, has been fenced off as a workman's yard, and the Jerusalem Municipality has begun storing rubbish bins in the south-west corner.

Even the few remaining graves are not safe: many of the headstones have been defaced or destroyed. "They have eliminated every stone here that has the name of the man inside, because they don't want anyone to claim them," says Abu Zahra. He believes that if the museum is built, it will not be long before the rest of Mamilla is appropriated by developers. "They will find a way to take more of the land, and step by step they will destroy the cemetery."

As we walked, he pointed out the shattered headstones of some of the tombs and translated some of the inscriptions on the few that remain intact. There was one commemorating the death of the "deceased martyr Ameen Abdelmo'ti Abu al-Fdel al-Alami, Sheikh and Imam" who died in 1346AH or "after Hejira" (AD1927), a reference to the Prophet Muhammad's journey from Mecca to Medina in AD581, which marks the beginning of the Islamic age.

Some people claim that Mamilla has graves dating to the era of Salah ah-din (or Saladin), who drove the Crusaders out of the Holy Land and recaptured Jerusalem, though archaeologists suggest most of them are no more than 400 years old. What no one disputes is that it contains the graves of sheikhs, imams, scholars, military leaders and members of the city's most important Arab families. "The name means 'a piece of heaven on earth', and it was a great honour to be buried there," says Raed Duzdar, whose ancestor is buried in the south-east corner of the plot, overlooking the site of the Waldorf Astoria hotel.

Ahmad Agha Duzdar was the Ottoman governor of Jerusalem between 1838 and the early 1860s. In 2005 the Turkish consulate helped Raed Duzdar renovate his grave. The tall, white stone, engraved with a red star and crescent and inscriptions in English and Arabic, was destroyed a few weeks later. All that is left of it is a few fragments of shattered stone.

Duzdar does not know who committed the act of vandalism, but he blames the authorities that allowed the SWC to develop the northern part of the site. "The government and the municipality say they're preaching tolerance, but they are allowing this ugly thing to be done to us in Jerusa­lem." He says that the sanctity of the cemetery is eternal. "No religion would accept the destruction of graves. It's very sinful."

Project stalls

Since the high court's verdict, Baskin has come up with various plans to stop the project. He was a signatory to another suit filed at the high court, claiming that the Israel Antiquities Authority, which prepared the site for construction, had misled the court about the number of burials it unearthed. Baskin has tried to persuade the Sephardi chief rabbi, Shlomo Amar, and his predecessor Ovadia Yosef, head of the ultra-Orthodox Shas party, to declare the site "unclean" because of the remains disinterred in the construction process.

Yet Baskin never held out much hope that any of the suits would succeed and began to fear that the museum was a "done deal". Yet last November, it seemed there had been an unexpected reprieve - work on the site appeared to stop, and the announcement that Gehry had left the project seemed to confirm it had begun to falter. The SWC maintains that it has spent the past two months removing pipes from the site. However, it has conceded that it is redesigning the project "to reflect today's world economic realities"; the budget has been cut from $250m to $100m, and the size of the complex has been halved. Rabbi Hier says that the SWC already has half the funds in place, and it will soon be holding a competition to find an Israeli architect to redesign the museum.

Gehry has denied that his decision to quit was prompted by "perceived political sensitivities", and Rabbi Hier refuses to acknowledge the anger over the destruction of Mamilla, saying that SWC members intend to "refocus all of our energies on bringing to Jerusalem, and the people of Israel, a project of crucial significance to its future". Baskin believes it will be a disaster if the rabbi succeeds, and yet, in some ways, the damage has already been done - no matter what happens next, the SWC will not be able to reinter the human remains dug from Mamilla. Nor will it be able to undo the offence it has caused the likes of Abu Zahra with its ill-considered attempts to spread "a message of tolerance between peoples".

Edward Platt is a contributing writer of the NS