The last Jews of Yemen are leaving. They are packing their bags and moving to Israel or the US. A community dating back to Biblical times is on the brink of extinction.

Sixty years ago one million Jews lived in Arab countries, but violence and state-sanctioned discrimination scapegoating them as Zionist spies have forced out all but 4,000 – who remain mainly in Yemen, Morocco and Tunisia.

Most Jews were airlifted from Yemen to Israel in the 1950s. The 400 left have resisted moving to Israel, having come under the influence of the non-Zionist Satmar sect. Some returned after a taste of life in the US or Israel (the government generally turns a blind eye to Jews travelling to the Jewish state). Now things have got so bad that even these die-hards are departing.

The murder in December of Moshe al-Nahari, a 30-year-old teacher based in Reda, north of the Yemeni capital, sparked this latest crisis. At first, the authorities claimed that the murderer was "mentally imbalanced". But it became clear that he was religiously motivated, screaming "convert or die, Jew!" as he pumped five bullets into his victim.

For some time jihadist gangs have been harassing Jews in Yemen. Girls have been abducted and forced to marry local tribesmen. Two years ago, 45 Jews, driven out of their village of al-Salem in north Yemen by threats from Shia Houthis, were relocated to the capital Sana'a.

Yemen is hardly an oasis of tranquility: it has more guns than people. The Jews are not the only ones to suffer in its long history of lawlessness and instability. Lately, however, Jews have had it especially tough.

Jews, tribal sheikhs, rights activists and lawyers all concur that harassment has reached an all-time high. After al-Nahari's murder, the Jews were besieged in their own homes and petrol bombs lobbed at them. Moshe's brother, rabbi Yahia Ya'ish, appealed to the government: "protect or deport us". Those wishing to leave could not claim their passports because the government's computers had mysteriously broken down.

Yemen's president, Ali Abdullah Saleh, pledged to take the Jews under his wing in Sana'a, where, in contrast to the countryside, he has firm control. Some say the government is well-meaning but ineffective; others that the promised relocation was never serious. The Jews were to be re-housed in two blocks, too cramped for their large families and vulnerable to attack. But they could not even sell their homes in Reda after local imams intimidated would-be buyers.

The Al-Nahari murder verdict in March was the last straw. During the trial the murderer's family threatened the victim's relatives. Instead of the prescribed death sentence, the judge ordered the murderer to pay "blood money". The Jews felt less secure than ever: the Jewish Agency and the US government swung into action to plan the Jews' rescue and resettlement.

Mahmud Taha, a journalist who has been following the story, is not surprised that the Jews want to go. "There is no option for the Yemeni Jews but to migrate. The local authorities have failed to protect them ... The Jews are fed up and have reached an intolerable situation," he said.

Mansour Hayel, a Muslim human rights activist and Yemeni Jewry expert, blames the government: "In Yemen there is hardly a mosque sermon that's free of bigotry. The government's own political rhetoric marginalises the Jews, and civil society is too weak to protect them," he says.

Perhaps because they understand that tolerance towards minorities is the key to strengthening Yemen civil society, Yemeni human rights activists

have been vigorously defending Jewish rights. They want the media to start promoting democracy and tolerance; and equal civil rights for Jews, who pay discriminatory taxes and, as dhimmis, suffer various handicaps under sharia law. But Jews whose lives are in danger are unlikely to stick around long enough to see such reforms implemented.

The lesson one draws from the final exodus of the Jews of Yemen is that the Arab world does not even tolerate non-Zionist Jews. There can be no future for the pitiful remnant in Arab lands if their safety cannot be guaranteed.

In Morocco, where the Jewish community is largest, Jews traditionally repaid the king's sympathy with tremendous loyalty. But the king of Morocco was unable or unwilling to prevent 260,000 Jews leaving in the face of rising antisemitism in the 1960s, media incitement and forced conversions.

Even benevolent rulers have been powerless to stem the rising tide of anti-Jewish hatred engulfing the Arab world. Few Arabs are now likely to meet a Jew in their lifetime, and the gullible believe the demonisation and conspiracy theories peddled by their media.

No wonder Jews have spurned official invitations for them to return to live in their countries of birth. Jews visit as tourists, but few see their future in these countries. In Tunisia and Morocco al-Qaida targeted Jews in 2002 and 2003. In April the murder of a Jew in Casablanca sent the community into a panic.In May, eight terrorists were arrested for planning attacks on Jewish sites.

If Morocco and Tunisia fail to keep a lid on jihadist terrorism and incitement, their last Jews, too, will soon be following the beleaguered Jews of Yemen into exile.