It is a cynical but time-honoured practice in Middle Eastern politics: the statesmen who decry the political and humanitarian crisis of the approximately 3.9 million Palestinians in the Israeli-occupied West Bank and in Gaza ignore the plight of an estimated 4.6 million Palestinians who live in Arab countries. For decades, Arab governments have justified their decision to maintain millions of stateless Palestinians as refugees in squalid camps as a means of applying pressure to Israel. The refugee problem will be solved, they say, when Israel agrees to let the Palestinians have their own state.

Yet in the two decades since the end of the Cold War, after two Gulf wars, and the rise and fall of the Oslo peace process, not a single Palestinian refugee has returned to Israel – and only a handful of ageing political functionaries have returned from neighbouring Arab countries to the West Bank and Gaza. Instead, failed peace plans and shifting political priorities have resulted in a second Palestinian "Nakba", or catastrophe – this one at hands of the Arab governments. "Marginalised, deprived of basic political and economic rights, trapped in the camps, bereft of realistic prospects, heavily armed and standing atop multiple fault lines," a report by the International Crisis Group (ICG) in Lebanon recently observed, "the refugee population constitutes a time bomb."

The fact that the divided Palestinian political leadership is silent about the mistreatment of the refugees by Arab states does not make such behaviour any less reprehensible – or less dangerous. Some 250,000 Palestinians were chased out of Kuwait and other Gulf States to punish the Palestinian political leadership for supporting Saddam Hussein. Tens of thousands of Palestinian residents of Iraq were similarly dispossessed after the second Gulf war.

Download the new Independent Premium app Sharing the full story, not just the headlines

In 2001, Palestinians in Lebanon were stripped of the right to own property, or to pass on the property that they already owned to their children – and banned from working as doctors, lawyers, pharmacists or in 20 other professions. Even the Palestinian refugee community in Jordan, historically the most welcoming Arab state, has reason to feel insecure in the face of official threats to revoke their citizenship. The systematic refusal of Arab governments to grant basic human rights to Palestinians who are born and die in their countries – combined with periodic mass expulsions of entire Palestinian communities – recalls the treatment of Jews in medieval Europe. Along with dispossession and marginalisation has come a new and frightening turn away from the traditional forms of nationalism that once dominated the refugee camps towards the radical pan-Islamic ideology of al-Qa'ida.

Daniel C Kurtzer, who has served as US ambassador to both Israel and Egypt and now advises the Obama administration, says that all American governments have resisted dealing with what he calls the most sensitive issue in the conflict – the normalisation of the status of the Palestinians – through a right of return to Palestine, or citizenship in other countries. "The refugees hold the key to this conflict's settlement," he says, "and nobody knows what to do with them."

In the unlikely event that President Obama's vision of a swift and final settlement of the Israeli-Palestinian conflict materialises, millions of Palestinians would still live in decaying refugee camps whose inhabitants are forbidden from owning land or participating in normal economic life. The only governing authority that Palestinians living in the camps have ever known is UNRWA – the United Nations Relief and Works Agency. Established by the UN on 8 December 1949 to assist 650,000 impoverished Palestinian refugees from the 1948 war, UNRWA has been battling budget cuts and strikes among its employees as it struggles to provide subsidies and services to Palestinian refugees, who are defined as "persons whose normal place of residence was Palestine between June 1946 and May 1948".

•••

The inclusion of the descendants of Palestinian refugees as refugees in UNRWA's mandate has no parallel in international humanitarian law and is responsible for the growth of the official numbers of Palestinian refugees in foreign countries from 711,000 to 4.6 million during decades when the number of ageing refugees from the 1948 Israeli war of independence in was in fact declining. UNRWA's grant of refugee status to the children, grandchildren and great-grandchildren of the original Palestinian refugees according to the principle of patrilineal descent, with no limit on the generations that can obtain refugee status, has made it easy for host countries to flout their obligations under international law. According to Article 34 of the UN Convention Relating to the Status of Refugees, "The Contracting States shall as far as possible facilitate the assimilation and naturalisation of refugees," and must "make every effort to expedite naturalisation proceedings" – the opposite of what happened to the Palestinians in every Arab country in which they settled, save Jordan. For all the easy criticism that can be levelled at UNRWA, it is hard to see how many Palestinian refugees would have survived without the agency's help.

The responsibility for the legal dimensions of their fate lies elsewhere, as UNRWA Commissioner-General Karen AbuZayd made clear at UNRWA's anniversary ceremony in New York on 24 September, before an audience that included Palestinian President Mahmoud Abbas and Queen Rania of Jordan – herself a Palestinian. "The protracted exile of Palestine refugees and the dire conditions they endure, particularly in the occupied Palestinian territory, cannot be reconciled with state obligations under the UN Charter," AbuZayd said. The result for the refugees, AbuZayd said at a forum the previous afternoon at the Princeton Club, is a "suspended state of existence" for which no one seems willing to accept political responsibility. The rest of the discussion, moderated by Ambassador Kurtner, made clear that anticipated solutions to the Palestinian refugee problem had failed to emerge – leaving a community in crisis.

"You can't ignore an entire people because it's awkward or inconvenient," says Dr Karma Nabulsi, a lecturer at Oxford and a former Palestinian representative at the UN. In the period immediately after Oslo, she added, Palestinian refugees in Arab countries hoped to be repatriated to areas governed by the Palestinian Authority. Today, despair has replaced that initial optimism. "What young Palestinian would want to resettle in Gaza or in the West Bank?" she asks.

Sharing a panel with Dr Nabulsi, the doveish former Israeli foreign minister Shlomo Ben Ami, who negotiated directly with Yasser Arafat at the failed Camp David meetings in 2000, asserted that Israel has suppressed narratives that would make clear its responsibility for the Palestinian refugee crisis of 1948. Indifference to the refugees' plight, he added, was shared by Israel's negotiating partner in the Oslo years – Yasser Arafat. "He was not a refugee man," Ben Ami said flatly. "He was much more centred on the question of Jerusalem. I heard him say to [Mahmood Abbas] in my presence, 'leave me alone with your refugees'."

Daily coronavirus briefing No hype, just the advice and analysis you need Enter your email address Continue Continue Please enter an email address Email address is invalid Fill out this field Email address is invalid Email already exists. Log in to update your newsletter preferences Register with your social account or click here to log in I would like to receive morning headlines Monday - Friday plus breaking news alerts by email Update newsletter preferences

It is no secret that certain Arab regimes saw the Palestinians under Arafat's leadership as an unwelcome occupation that stripped Jordan bare and destroyed Lebanon. Similarly, Arafat often used the threat of destabilisation and assassination to get Arab regimes to fund the Palestinian cause. Still, the record of Arafat's Palestinian Authority in its territories during the 1990s attests to the truth of Ben Ami's observation, which applies both to Arafat's Fatah and to Hamas. Despite $10bn in foreign aid, not one refugee camp in the West Bank or Gaza has been replaced by modern housing. On the West Bank, chances for normal Palestinian communal life have been shattered by Israeli settlements, arrests, checkpoints and roadblocks, and by 15 years of abuses by Fatah. Even under the best of circumstances, an influx of refugees would further destabilise a Palestinian economy that is kept afloat by the world's highest per capita receipts of foreign aid.

Daniel Kurtzer agrees no one is likely to make a deal that includes a substantial return of the Palestinian diaspora. "Most Palestinian refugees know it, as do the settlers," he says. So rather than wait for American mediators or Arab states to impose solutions on them, the Palestinians themselves should begin to tackle the diabolically difficult issues inherent in the resolution of their political and economic future. "What we need is a refugee summit," he says. "I'm looking for a real conversation that must start internally and soon."

After 60 years of failed wars, and failed peace, it is time to put politics aside and to insist that the basic rights of the Palestinian refugees in Arab countries be respected – whether or not their children's children return to Haifa anytime soon. While Saudi Arabia may not wish to host Israeli tourists, it can easily afford to integrate the estimated 240,000 Palestinian refugees who already live in the kingdom – just as Egypt, which has received close to $60bn in US aid, and has a population of 81 million, can grant legal rights to an estimated 70,000 Palestinian refugees and their descendants. One can only imagine the outrage that the world community would rightly visit upon Israel if Israeli Arabs were subject to the vile discriminatory laws applied to Palestinians living in Arab countries. Surely, Palestinian Arabs can keep their own national dream alive in the countries where they were born, while also enjoying the freedom to work, vote and own property?

A practical solution to the crisis of the Palestinian refugees in Arab countries will focus on Lebanon, Syria and Jordan, which together play host to approximately 3 million of the estimated 4.6 million Palestinian refugees living outside the West Bank and Gaza. While each of these countries has chosen different legal and political approaches to the 1948 refugees and their descendants, they share a political desire to sublimate the rights of Palestinian residents, treating them as unwanted guests or as tools to be used in pursuing wider political interests – but rarely as fully-fledged members of society. Lebanon, where Palestinians led by Yasser Arafat are widely blamed for having sparked the 1975 civil war, is the worst offender against international norms. Yet even in Jordan, which is in many ways a model for the humane treatment of a large refugee population, Palestinians today feel markedly less secure than they did two decades ago, or even five years ago.

•••

Outside of Iraq, whose Palestinian population fled en masse after the fall of Saddam, nowhere has the situation of the Palestinian refugees worsened so dramatically as in Lebanon. Since the early Sixties, Palestinians there have been barred from working in medicine, dentistry and the law. In 2001, the Lebanese parliament adopted an amendment to the country's property laws that prohibited the acquisition of real estate by "any person not a citizen of a recognised state" – meaning the estimated 250,000 to 400,000 Palestinians living in Lebanon. Palestinians who had acquired real estate prior to 2001 were barred from bequeathing property to their children.

Right-wing Christians and Shi'ite radicals alike support discriminatory legislation that further impoverishes Palestinian refugees in Lebanon, with the stated goal of preventing them from beginning the process of naturalisation, known as tawtin. In his inaugural speech in May, 2008, Lebanese President Michel Suleiman, a Christian and former head of the country's armed forces, reaffirmed "Lebanon's categorical refusal of naturalisation", a statement echoed by the former Lebanese ambassador to the US, Nassib Lahoud, who told us recently in Beirut: "The confessional balance does not allow these things to happen ... at the moment the Palestinians are citizens of a state that does not exist." His sentiments were echoed by Hizbollah's spokesman on the Palestinian question, Hassan Hodroj. "The threat of tawtin is genuine," Hodroj explained. "It is one of the ways in which Israel, backed by the US, is endangering the region."

The fact that the living standard of Palestinian refugees in Lebanon has been deemed "catastrophic" by both UNRWA and by the Lebanese government can therefore be understood as a deliberate result of official state policy that is supported by all parties across Lebanon's divided confessional spectrum. As a member of the Lebanese parliament, Ghassan Moukheiber, explained in an interview with the ICG, "our official policy is to maintain Palestinians in a vulnerable, precarious situation to diminish prospects for their naturalisation or permanent settlement".

Yet the results of this horrifying policy may not be confined to Palestinian refugee camps in Lebanon. In his book Everyday Jihad, about the experience of refugees in the Ain al-Hilweh camp, home to an estimated 70,000 Palestinians, the French scholar Bernard Rougier describes the results of decades of exclusion and marginalisation which have severed the refugees from any connection to a lost homeland – or the country in which they were born. As a result, he says, many Palestinians have abandoned a failed nationalism for the radical millenarian ideas associated with al-Qa'ida. "Palestinian salafist militants have devoted themselves to defending the imaginary borders of identity," Rougier writes, "declaring themselves the protectors and guardians of the cause of Sunni Islam worldwide."

Visitors to the Ain al-Hilweh camp are immediately made aware that they have entered another world. While Lebanese army checkpoints ring the camp, the Lebanese state has no presence inside. Food, water and other basic services are provided by UNRWA, while armed factions openly display weapons in muddy alleyways and recruit generations to serve under their banners. It is easy to see why the secular promise of Palestinian nationalism has faded and why the promise of a Muslim paradise without borders might take its place. One of the 9/11 hijackers dedicated a poem to Ain al-Hilweh's most prominent jihadist in his videotaped will, and dozens of Palestinian fighters from the camp joined al-Qa'ida in Iraq.

"The situation is the camp is deteriorating," Rougier told us, when we asked about whether things were getting better or worse for the Palestinians of Lebanon. Bound by their absolute opposition to tawtin, he says, Lebanese leaders are creating a radicalised Palestinian population that will eventually have to be absorbed into Lebanon, despite having little or no allegiance to the state.

Sahar Atrache, lead author of the ICG report on the Palestinian refugees in Lebanon, agrees. "Palestinians refugees in Lebanon lack means of socio-economic advancement and are bereft of hope," he says. "They are vulnerable on all counts – politically, legally and above all physically. The status quo is good neither for the refugees nor for Lebanon itself."

•••

While Palestinian refugees and their descendants inside Syria are not allowed to vote or hold Syrian passports, they are free from the overt discrimination that has turned Lebanon into a recruiting ground for al-Qa'ida. The legal status of Palestinians inside Syria is defined by a 1956 law that states that grants them "the right to employment, commerce, and national service, while preserving their original nationality". More than 100,000 of the estimated 450,000 Palestinians in Syria live in or around the Yarmouk refugee camp, which long ago became a neighbourhood of Damascus.

While Palestinians are reasonably well integrated into the Syrian socio-economic structure, according to the scholar Laurie Brand they do not have the right to vote, nor can they stand for parliament or other political offices. Palestinians are barred from buying farmland and prohibited from owning more than one house. The female descendant of a Palestinian refugee can become a Syrian citizen by marrying a Syrian man. The male descendants of Palestinian men and their children are barred from acquiring Syrian citizenship, even if they marry Syrian women.

The major focus of Syrian interest in the Palestinian refugees has long been as an extension of the Assad regime's policy towards its neighbours – Israel and Lebanon. Damascus has long hosted a variety of Palestinian terror groups that rejected the Oslo process, including Ahmad Jibril's Palestinian Front for the Liberation of Palestine-General Command (PFLP-GC). More significantly, Damascus is also the political and logistical centre for Hamas. "Syria's support for armed Palestinian groups is key to pressuring Damascus' neighbours, most notably Israel and Lebanon," says Andrew Tabler, author of the Syria-watching blog Eighth Gate.

Syria increases its leverage inside Israel by weakening Fatah and strengthening Hamas. In Lebanon, Syrian military and political interference has turned the refugee camps into "security-free islands" (juzur amniya) where bombers can be recruited, bombs manufactured, and plots can be directed beyond the reach of the Lebanese army and police. "Life for the Palestinians was deliberately frozen for political manipulation," concludes Lebanese analyst Tony Badran. "Syria has no interest in normalising that situation."

While Syria imposes a measure of security on its Palestinian neighbourhoods, it foments insecurity and violence in Lebanon and Gaza, splitting the Palestinian polity and fuelling the misery of Palestinians throughout the region.

•••

Jordan is the only Arab nation that has integrated large numbers of Palestinians as full-fledged citizens. This is due not only to the unification of the East Bank and West Bank of the Jordan River valley under Hashemite rule between the creation of the State of Israel in 1948 until Israel's occupation of West Bank in 1967, but also to the luck of having had an enlightened monarch committed to the compassionate treatment of the estimated 100,000 refugees who crossed the Jordan River during the nakba in 1948. Israel's occupation of the West Bank following the 1967 war triggered a second exodus of 140,000 refugees into Jordan.

Today, almost 2 million of Jordan's 6 million people are registered Palestinian refugees, the largest concentration of current and former refugees in the Palestinian diaspora – and increasingly, tensions have deepened between the Palestinians and the "East Bank" establishment. This summer in Amman, ambiguous declarations by the recently appointed minister of the interior, Nayef al-Kadi, who is widely perceived to be anti-Palestinian, led many Jordanians of Palestinian origin to fear they would be stripped of Jordanian identity numbers. Speaking to the London-based Arabic-language newspaper Al Hayat, al-Kadi confirmed that some Palestinians would be stripped of citizenship, ostensibly to counter Israeli plans to turn Jordan into Palestine. "We should be thanked for taking this measure," he said. "We are fulfilling our national duty because Israel wants to expel the Palestinians from their homeland." Panic about their status spread quickly among the Palestinian community.

In interviews this month, senior Jordanian officials sought to quell such fears, while also suggesting there was at least some substance to al-Kadi's explosive suggestion. Faisal Bakr Qadi, the director of the Interior Ministry's office of Inspections, said Palestinians in Jordan were not being systematically stripped of citizenship. Rather, he explained that the government's current review of Palestinian national status dated back to 1988, when King Hussein, in response to demands by Palestinian and Arab leaders, disengaged administratively from the West Bank. Palestinian refugees, he said, meaning those who came to Jordan in the 1948 exodus, were to remain "full Jordanian citizens". "Displaced" Palestinians, or those who had come in 1967 and afterwards, would be able to maintain their yellow identity cards and numbers and de facto citizenship, provided they returned to the West Bank to renew the Israeli passes that permit them to go back and forth between Jordan and the West Bank.

Since 1983, he said, Jordan had given the coveted yellow cards – which enable Palestinians to work without special permits, pay local tuition rates in school, and enjoy full government services – to 280,000 Palestinians, whereas it had "frozen" the cards – or downgraded their status – of only 15,856 people. So far this year, he said, 9,956 cards were upgraded, 291 downgraded.

While many diplomats doubt these numbers, Jordanians insist there is no plot or plan to expel or deny citizenship to Palestinians who have lived virtually their entire lives in Jordan. "We want to ensure that when and if the peace process succeeds in establishing an independent Palestinian state, Palestinians living in Jordan will be in a position to choose their citizenship by having their status in order in both Jordan and Palestine," said an official close to King Abdullah.

Yet the distinctions that seem meaningful in Amman are not clear to some of the almost 94,000 Palestinian residents of Baqa'a, the largest of the 10 official refugee camps run by the UN. Some Palestinians in Baqa'a complain about the "new regulations" and the lack of identity cards that enable them to work without special permits and educate their children in public schools. Anxiety about the future pervades this ramshackle suburb at the northern edge of Amman, which began as an emergency relief centre after the 1967 war and is now a sprawling mini-city with its own basic shops, shawarma (sandwich) stands, and services. Many of the people we spoke to claimed that they knew someone, or had a relative, neighbour and friend whose identity card had been revoked, or whose status had inexplicably been changed.

For many of these refugees at the bottom of Jordan's social and economic pecking order, life without papers means hiding from the police who constantly patrol their camp's streets, being too poor to send any of your eight to 10 children to college, a lifetime of menial labour, and only a threadbare dream of returning to a homeland that most of them have never seen. There is strong suspicion of the state, but also of their neighbours, who are divided into "'48 people" and "'67 people". "Some of the newcomers would give away Al Aqsa for a Jordanian identity card," says Heba, a mother of eight, mentioning Islam's celebrated mosque in Jerusalem, one of its holiest shrines.

"We're Jordanians," says her son, Mustapha, a slender, 20-year-old in a bright orange T-shirt emblazoned with meaningless words in unknown languages. "This is the best place in the world," he says, pointing around the bare living room whose worn rugs and threadbare pillows cover the floor on which he and all his siblings sleep. "We would never leave here. But I'm loyal to my country, and I would like to visit it one day."

He seems perplexed when asked which is his country – Jordan or Palestine. "We have no security here, but we are Jordanians," replies Mustapha, who lounges on a mattress in a two-storey cement house down the road while one of his five daughters offers tiny glasses of steaming herbal tea and cardamom-scented coffee. "Everything I have is here. This house. My car. My job. What would I have in Nablus or Be'ersheba?" he declares. "My children know nothing but Jordan. And we will stay here."

That determination, echoed repeatedly through the dilapidated cement homes that line Baqa'a's gravelly streets and dust-filled shops, is precisely what terrifies Jordan's East Bank establishment. Jordanians have reason to fear their Palestinian guests. Many Jordanians have not forgotten "Black September", the civil war launched by Arafat's Fatah organisation in 1970 which nearly toppled King Hussein's kingdom.

Moreover, having grown accustomed to their near monopoly on jobs provided by the government, Jordan's largest employer, Jordanians fear demands for political equality from Palestinians, most of whom would probably choose to remain in Jordan, relinquishing their "right of return' in favour of compensation. An end to the Palestinian-Israeli conflict would surely threaten Jordan's informal division of power: East Bankers dominate the army, the security services and most civil-service posts, while Palestinians are disproportionately represented in business. Palestinians may advise the king in the royal court, but there has been only one Palestinian prime minister, who served for eight months. Palestinians now comprise only 23 of Jordan's 110 MPs.

"The closer we get to a solution," says Adnan Abu Odeh, a Palestinian who was one of King Hussein's royal court chiefs and also held other important government posts, "the more anxious society becomes. We are approaching a moment of truth."