Outside the window of Tim Mackintosh-Smith’s home in Sana’a, the Yemeni capital, are reminders of the long sweep of Arab history – child soldiers mourning martyrs of the country’s ongoing war, rocket salvoes, sectarian rivalries, hypnotic slogans and a mosque dating back to the seventh century and the rise of Islam in the Arabian peninsula.

The view is simultaneously rich, bleak and thought-provoking: for three millennia, dynasties have come and gone, from the Sabaeans and Himyaris to the Umayyads of Damascus and the Abbasids of Baghdad. Later came the al Saud – the family that gave its name to a still powerful kingdom. Interactions between desert (badu) and town (hadar), semi-nomadic tribes and settled peoples, strong men and weak institutions, are a constant theme. Language, faith, and loyalty come together in complex and far-flung combinations.

Arabs retells a familiar story in unexpected ways. It focuses first on the social and economic changes (the domestication of the camel was key) that shaped the pre-Islamic world before the transformation that began in Mecca in AD622. Perfumes and gems were the precursors of the petroleum and gas of modern times. In the background were always challenges from Assyrians, Persians, Romans and Mongols, narrated and fought by a colourful cast of oracles, orators and commanders of dogged, lightly equipped horse-mounted warriors.

Mackintosh-Smith is an unusual Englishman abroad: a writer who lives, as he puts it, in a land not a library, experiencing history in situ. He combines deep learning with penetrating insights delivered with dazzling turns of phrase and illuminating comparisons.

The author’s acclaimed Travels in Dictionary Land was a clue to his fascination with both his adopted country and the Arabic language – the golden thread that runs through this wonderful new book. “Of the three outstanding conquests of Arab history – those of arms, Islam, and of Arabic – the first and most enduring has been the victory, over themselves, of the tongue that bears their name,” he writes. “The grammar of their [the Arabs’] history would be unstoppably active, and they would earn not just a capital letter but a definite article.”

The first known inscription mentioning Arabic dates back to 853BC – 1,400 years before the boy Muhammad was recognised as a prophet. In the conquests that followed, disparate tribes and peoples from the Gulf to north Africa were united by the swirl of Arabic script and that most haunting of Muslim prayers “La ilaha illa Allah”: “There is no god but Allah.” The first Arabic book, the Qur’an, was “not only the scripture of Islam, but the founding text of Arabdom… with all the historical weight of a Pentateuch, a Magna Carta and a Declaration of Independence”. Arabic and Islamic history, he argues, in a characteristically succinct take on a long debate, “are conjoined not just at the hip, but at the heart”.

Links and connections over time and distance are amazing: on a visit to Mauritania – the furthest extreme of the Arab(ic) world – the author hears the familiar rhythms of a Yemeni dialect, a living connection to a hijrah (migration) that took place half a millennium ago. Spain’s 4,000 Arabic loan words are a monument to seven centuries of rule. Less familiar is his admittedly unproven but delightful idea that the English hunting cry of “tally-ho” is related to the (far more guttural) Levantine Arabic imperative for “come here”.

Women sit before Fez, Fes el Bali in Morocco. Photograph: Alamy

Mackintosh-Smith has an enviable ability to enrich the big picture with fascinating detail and telling parallels. Arab power reached a peak in the ninth century, when the Abbasid dinar was the equivalent of the US dollar and intellectual creativity was at a peak, as attested by Arabic words like alcohol, algebra and algorithm. The spread of the “new Latin” was accelerated by cheap paper while Europe still relied on parchment.

Beautiful but print-unfriendly Arabic fell behind, however, when movable type developed during the Renaissance. The gap narrowed in the 19th century and in the digital age, but the messages delivered by today’s autocrats now dominate our age of “mis-information technology”. Twitter wars are as much a part of the Arabian landscape as the graffiti scrawled on pre-Islamic tombstones.

Interactions with the empires of others have always been key to Arab history – “caught on their rock between predatory powers”, as he puts it. Napoleon’s arrival in Egypt in 1798 brought modern technologies and fashions and an age of nationalism that was propagated by Christians like the Lebanese Ibrahim al-Yaziji. Britain’s role, during and after the first world war, was to leave a lasting imprint from the Sykes-Picot agreement to the Balfour Declaration fatefully promising a Jewish national home in Palestine. “Imperialists certainly divided and ruled,” Mackintosh-Smith observes judiciously, “but more often than not they were driving wedges into old splits.”

What he calls the Arab “age of hope” was short-lived and deeply disappointing. It was embodied by the charismatic, matinee-idol figure of Egypt’s Gamal Abdel Nasser, who humiliated Britain during the Suez crisis, though the great diva Umm Kulthum was equally if not more important than her nationalist president. Radio Cairo, in those days, was literally the voice of the Arabs.

In the 21st century, continuities still abound: US-imposed regime change in Iraq was one thing, but “dictionary change” – what the idea of freedom means – was another matter, as the spirit of Saddam Hussein still preoccupies “suq soothsayers and casbah Cassandras”. In Raqqa, northern Syria, a tax imposed on Christians in 640 was reinstated by Islamic State in 2014. Actual nomads on camels have been replaced by Toyota trucks mounted with heavy machine guns – but the rampage by camel-borne regime loyalists in Cairo’s Tahrir Square in 2011 was a potent image that established a grim link between past and present.

Bringing the story up to date, Tunisia, oriented northwards towards the Mediterranean and Europe, is the only (relative) success story of what was over-optimistically dubbed the Arab spring. Egypt is widely considered to be more oppressive today than it was under the deposed Hosni Mubarak. Gulf sheikhs, with their “conspicuous constructions” of skyscrapers and shopping malls, have become monarchs as absolute as any in human history, Mackintosh-Smith concludes. Syria lies in ruins. Some see only a long winter ahead.

Yet even viewed from Sana’a, not all is lost. Hope is kept alive by a writer who lifts spirits and fires the imagination by comparing pre-Islamic poetry to pop festivals and quotes the timeless reflections of a sixth century Arabian for the loss of his young son Zayid – foreshadowing the famous lament, at the going down of the sun, and in the morning, when we too remember our loved ones. Words are, indeed, still the sharpest weapons.

Ian Black is a visiting senior fellow at the Middle East Centre, LSE and a former Guardian Middle East editor, European editor and diplomatic editor. His latest book is Enemies and Neighbours: Arabs and Jews in Palestine and Israel, 1917-2017.

• Arabs: A 3,000-Year History of Peoples, Tribes and Empires by Tim Mackintosh-Smith is published by Yale University Press (£25). To order a copy go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £15, online orders only. Phone orders min p&p of £1.99