There are few things more profoundly dead than an ex-empire, but around the time that the Soviet empire came apart at the seams, I became aware that the ghosts of a much older one – that of the Turkish Ottomans – were still haunting its former domains.

It was in the spring of 1990. All Europe's communist dominoes had already fallen over, the most recent being Romania, whose dictator Ceausescu had just been executed. The only one left standing was tiny, reclusive Albania. Every half-serious newspaper in Fleet Street wanted a bite of it, but foreigners were barred from entering – not only journalists, but even ordinary tourists. The only outsiders admitted were archaeology enthusiasts who were occasionally permitted to undertake study tours.

And so it came to pass that the next scheduled archaeological study tour was strangely over-subscribed. Of the 20 or so who signed up for it – claiming a range of occupations from farmer to advertising copywriter to ballet dancer – almost all were journalists, as our unlucky tour guide soon discovered. The exceptions were a clean-cut couple who turned out to be professional Christians, and four Pakistanis from Dewsbury. All six shared a common mission: to restore the faith – variously Christian and Muslim – to atheist Albania.

Download the new Independent Premium app Sharing the full story, not just the headlines

I had no idea that Islam had got as far into Europe as Tirana, let alone that its embers had survived being stamped on for many years by Enver Hoxha, Albania's communist dictator. But it was in Albania's grim and impoverished streets that I got my first whiff of the Ottomans: the beguiling reek of Turkish tobacco and coffee, the pungent kick of slivovitz. What we wrote about were the country's modern hallmarks, the horrible housing estates and ubiquitous concrete bunkers. But what took one by surprise and lingered in the memory were the intricate old lanes of a town such as Gjirokaster, the compact but handsome old stone mosques built, one learnt, by colonisers from Istanbul for whom Albania was just as dismal a backwater as it was for us.

Albania was on the frontier, as much for the Ottomans as for the Russians and later for the EU. And as Eastern Europe slowly emerged during the 1990s from its communist purdah, one discovered that the unique fragrance of Ottoman civilisation lingered much closer to home. Greece, every corner of disintegrating Yugoslavia, Bulgaria, Romania, even Ukraine – all had been under Turkish control for centuries. They had been conquered and colonised by the Ottomans – by Suleiman the Magnificent (born, 1492; died, 1566) to be precise – before the English had gone anywhere.

And what makes us scratch our heads in puzzlement about that vanished empire is that, although the Ottomans were Muslims, and the figure we think of as the Muslim pope, the caliph, was identified with Istanbul, large populations of Christians and Jews continued to live and prosper right across the Ottoman Empire.

After 150,000 Jews were expelled from Spain in 1492, they were formally invited to make new homes inside the Ottoman Empire. True, the infidels faced higher taxation than the faithful, but for centuries, the Ottoman version of Islamic rule was distinguished by pluralism and peaceful coexistence. The now sadly beleaguered and diminished Christian communities of Syria and Egypt and Iraq bear witness to that.

That is one of the reasons why BBC2's new series, The Ottomans – Europe's Muslim Emperors, fronted by Rageh Omaar, has such a timely feel. We know so little about this extraordinary dynasty; a single family ruling an empire that at its peak encompassed half of Europe and most of north Africa and that lasted from before the Peasants' Revolt to the age of aviation – and most of what we know is wrong.

We vaguely recall that old Ottoman bogeyman: the savagery of the moustachioed Turk, the two Sieges of Vienna when the shadow of the prophet fell across the whole of Europe. Just as vaguely we know of the luxurious, exotic aspect, the emperor's harem with his Christian concubines, the hookah, that whole lush, sensual, decadent world summed up by the single word "Oriental". But the far richer and more interesting reality eludes us.

The Turks came swarming across the Anatolian plain like so many other fierce, rootless, horse-borne central Asian nomads both before and after them. But when these predators got down off their horses and started to learn new things from the people who resisted them, they began to amount to more than the destruction they wrought. In the case of the Turks, they converted to Islam then ran up against the Byzantines in what was then called Constantinople, the eastern arm of the divided church of Rome.

They exchanged the swooping, looting and slaughter of their nomadic past for more polite modes of competition as they ate into the declining Byzantine possessions, taking their rulers as "honorary hostages", forcing them to create a Turkish quarter inside the capital. It was a long game of wits, strength and cunning, during which the invaders gradually absorbed many of the Byzantines' civilised ways. The fatal blow was struck not by the Turks but by the treacherous Catholics of the Fourth Crusade, who occupied the city in 1204, fatally weakening the empire and hastening its final overthrow two-and-a-half centuries later.

Thus began the Ottomans' golden age, which stretched from the end of our Hundred Years' War to the declining years of Charles II. Under a succession of brilliant emperors culminating in the extraordinary Suleiman, they conquered half of Europe and the whole of north Africa bordering the Mediterranean as far as Morocco. They also gained control of the Islamic holy sites of Mecca and Medina and, after the conquest of Egypt, the Ottoman sultans adopted in addition the title of caliph, asserting their leadership of all Muslims.

Suleiman was rightly called Magnificent not only on account of his vast turban and his conquests of Hungary and Belgrade, but also because he was one of those rare warriors who was many other things besides: an artist, a great patron of the arts, and a poet. But it was the name by which he was known within the empire, Kanuni Suleiman, which is most relevant and resonant today: Suleiman the lawgiver.

The Ottomans were bound by Sharia, but Suleiman understood how inadequate that archaic charter was to regulate the affairs of a modern empire. The act of theft, for example, under Sharia involves simply placing a hand inside another's property and removing something, while more sophisticated forms of theft such as counterfeiting, let alone copyright violation, were outside its norms. Suleiman presided over a new system of laws which respected Sharia while taking account of the developing nature of Ottoman civilisation – a legal system that endured for centuries.

As with other empires both before and since, it is easy to sentimentalise the Ottoman centuries. There was no "multiculturalism" in our sense: there was no doubt about which religion was on top. The stability of the administration rested on the barbaric custom of kidnapping Christian boy children from the Balkans year after year, converting them to Islam, and turning them into soldiers or civil servants, depending on their gifts.

In either case, their orphaned condition meant that they had no ambitious relatives waiting in the wings, threatening the power of the ruling family, and the Sultan advanced them according to their abilities, regardless of how humble were their origins. "The shepherd who rose to become an illustrious grand vizier was a figure that never ceased to fascinate European observers," wrote one authority. Thus on the foundations of a brutal tradition was built Europe's first functioning meritocracy.

But the real tragedy of the Ottoman Empire is not the way they ruled, but what happened after they left. Everywhere one looks, those lands are, or have recently been, in flames, from Bosnia to Iraq and from Tunisia to Syria. And wherever one looks, that precious Ottoman legacy, the cohabitation and interdependence of different faiths, is under mortal threat.

If toleration of other faiths became a hallmark of Ottoman rule, it was not obvious in the empire's early days. Pope Benedict XVI, now in retirement, brought the wrath of the Islamic world down on his head in September 2006 when, during a scholarly sermon at the University of Regensburg, he quoted comments by the late Byzantine emperor Manuel II Paleologus, who became an "honorary hostage" of the Ottomans in the 14th century.

The Christian emperor chastised Islam for its reliance on the sword: "Spreading the faith through violence is something unreasonable," he said. "Violence is incompatible with the nature of God and the nature of the soul… Show me just what Mohammed brought that was new and there you will find things only evil and inhuman, such as his command to spread by the sword the faith he preached." Manuel's comments were understandable in the context of the 14th century and they are unpleasantly resonant again today, in countries from Pakistan to Kenya and beyond, where Muslim zealots appear to believe that the rule of terror is pleasing to God.

But they would not have been correct during the Ottomans' long heyday, when Christians, Jews and other minorities practised their beliefs openly and without fear. And because the Ottoman sultans were also recognised as caliphs, the supreme religious as well as political leader, throughout the Islamic world, that model of peaceful co-existence was for many centuries the norm.

In fact, the rise of fanaticism can be traced directly to the end of the caliphate in 1923. Kemal Ataturk, the Turkish hero of Gallipoli, buried the Ottoman Empire with little ceremony when he took power after the First World War, abolishing all its symbols, from Arabic script and the veil to the fez. The caliphate was, in his nationalist, secularising view, just another antiquated hangover from the long, humiliating decline of what had been known for nearly a century as "the sick man of Europe".

But the caliphate's abolition had an impact far beyond the borders of Turkey: it removed a central point of reference for Muslims everywhere. The poet WB Yeats was not thinking of Islam when he wrote his famous lines: "Things fall apart/ the centre cannot hold," but that was the effect of the caliphate's downfall.

It was especially devastating far away in India. Up to that point, Hindu and Muslim freedom fighters had been in lockstep against British rule. With the caliphate gone, Muslims suffered a fatal blow to their prestige, as well as the removal of the ancient source of religious authority. Within a few years, the Muslim League had parted company with Congress and was demanding a separate state for the subcontinent's Muslims, which two decades later (on 14 August 1947, to be precise) became Pakistan.

Other consequences of Ataturk's decision continue to roll around the world. Afghanistan's Taliban are as far removed from the suaveness and sophistication of the Ottomans as one can get: many of their leaders were illiterate, and the rule they imposed was notoriously intolerant and rigid. Their interpretation of Sharia was brutally reductive, as if the only way to righteousness was to recreate the conditions of life that prevailed while the Prophet was still alive.

What gave them their confidence to impose such brutality on the poor, suffering Afghans? It is the fact that there was no figure of authority to challenge them. To justify his rule, their one-eyed leader, Mullah Omar, usurped the title of caliph – Caliph of the Emirate of Afghanistan. One of the symbols of legitimacy of the Ottoman caliphs was a cloak allegedly worn by the Prophet, kept locked away in an ornate chest in Istanbul; one day Omar, standing on a Taliban pick-up truck, produced a Prophet's cloak of his own, though how it had survived 1,500 years in such excellent condition was not explained.

The Ottomans, absolutist rulers though they were, developed laws that both Muslims and Christians could live by, and sustained a model of Islamic moderation which prevailed until the caliphate's destruction. For that reason they are much missed. Since then, from Kandahar to Somalia and from Tora Bora to Mali, as Yeats wrote: "mere anarchy is loosed upon the world":

"The blood-dimmed tide is loosed, and everywhere

The ceremony of innocence is drowned;

The best lack all conviction, while the worst

Are full of passionate intensity…"

'The Ottomans – Europe's Muslim Emperors' airs on BBC2 from 6 October