In 2001, in a violent attempt to advance the cause of Islamic fundamentalism, a clutch of men empowered by the Taliban brought down a titanic pair of structures that loomed over their skyline. No lives were lost. The few people living near the Buddhas of Bamiyan, in central Afghanistan, were cleared out first, before anti-artillery weapons were trained on the sculptures, carved out of the russet cliffs of the Bamiyan valley. "These statues have been and remain shrines of unbelievers," a February 2011 edict from Mullah Omar had proclaimed. Their destruction was carried out with a rare and perverse vim. Failing at first to pulverise the Buddhas, the Taliban called in Pakistani and Arab engineers to finish the job. In The Places in Between, Rory Stewart observed that the Taliban had scorched a fresco on the ceiling of one of the caves that honeycomb the cliffs and then stamped boot-prints over the patina of soot. "This must have taken some effort, as the ceiling was 20 feet high."

The Buddhas had stood for a millennium and a half; the smaller figure, 38m tall, was built around AD550, and the larger – at 55m only a little shorter than London's Monument – around AD615. In The Buddhas of Bamiyan, Llewelyn Morgan, a lecturer in classics at Oxford University, explores not so much the heartbreaking demise of the statues as their remarkably long lives. How and why did the Buddhas survive more than a dozen centuries of an Islamic Afghanistan, only to meet their end at a particular political moment in 2001? The final downfall of these sculptures – their arms already snapped off, their surfaces pitted by erosion and minor vandalism – represented the nadir of a long and complex process of civilisation. In the plangent words of the Iranian filmmaker Mohsen Makhmalbaf, perhaps the Buddhas could take no more: "Even a statue can be ashamed of witnessing all this violence and harshness happening to these innocent people and, therefore, collapse."

The story of Bamiyan, Morgan suggests, is really the story of Afghanistan itself – of a fractured land with the misfortune of being one of the world's great crossroads, the benefits accruing to it from trade and commerce rubbed out by the curse of being coveted for its strategic location. Bamiyan lay on a branch of the silk route that cut efficiently through the heart of the Hindu Kush mountains, providing both merchants and soldiers access to the Indian subcontinent, to China, to central Asia and thence to Europe. It has hosted a multitude of nationalities, religions and armies, a tinder-dry mix ever primed to be set afire: Greek stragglers from Alexander's campaigns; Hazaras descended from Genghis Khan's troops; Indians and Pashtuns and Persians and Turks; Buddhists and Christians as well as Shia and Sunni Muslims; the forces of the British Raj, the Soviet Union, the Taliban and Nato. Incredibly, through this tumult, Bamiyan managed to retain an air of pacific calm; the historian Arnold J Toynbee, visiting in 1960, wrote of "peace in the glistening white poplar-trunks … peace in the shadowy shapes of the Buddhas and the caves".

Buddhism arrived in the Bamiyan valley in the first or second century AD. This was as far west as the religion would advance, but it flourished here; archaeologists have discovered the remains of a great stupa – a domed home for Buddhist relics – and the Chinese traveller Xuanzang, who passed through Bamiyan in AD629, wrote of "several tens" of monasteries with "several thousand monks". An uncommon stability prevailed in Bamiyan at the time, the result of a delicate balance of regional powers. Morgan proposes that Buddhism benefited from this stability, but also from Bamiyan's nature as a hub of commerce. The town's monks, Xuanzang noted, shrewdly charged visitors to see their relics, and their monasteries functioned as dormitories, bazaars and banks for merchants. Out of such unexpectedly mercantile zeal were Bamiyan's giant Buddhas funded.

Nearly all of Morgan's material is distilled from the recorded impressions of travellers such as Xuanzang, journeying through Bamiyan en route elsewhere. In the book's latter sections, which draw on the writings of surveyors, soldiers and antiquarians of the Raj, these sources tell us little that is new or noteworthy about Bamiyan or Afghanistan. But similar texts by Muslim travellers allow Morgan to parse the surprising malleability, over the ages, of Muslim attitudes towards this Buddhist iconography.

The contrasts between these attitudes is striking evidence that the Taliban were by no means acting in a "typically" Islamic manner in razing the statues. The emperor Babur, who founded the Mughal dynasty and ordered several Indian idols to be destroyed, didn't mention the Buddhas in his accounts of his travels through Bamiyan. Khan Abdul Gaffar Khan, a Pashtun and a 20th-century freedom fighter in India, called the Buddhas "an unparalleled example of perfection in the art of sculpture". In 1842 a British prisoner of Afghan soldiers recalled his captors firing idly at the statues and "cursing them as idols"; on the other hand, a 12th-century Islamic text titled The Wonders of Creation marvelled at these "talismanic" images and declared that "the Creator gives divine inspiration to His subjects to make such wonders". Tellingly, a member of a delegation of 11 Muslim clerics who pleaded with Mullah Omar to cancel the demolition, would later say that the Taliban "had no knowledge about Islam. They are so naive, they really can be influenced."

This textured response of Islam to Bamiyan's Buddhas – and even, on occasion, the projection on these statues of Islam's own folkloric figures – protected them from significant harm until the 21st century, Morgan argues. Surprisingly, he misses another crucial part of the explanation. While Sunni doctrine is rigidly iconoclastic, the Shia branch of Islam – long dominant in the Bamiyan valley – has been considerably more tolerant of imagery used in the service of religion. This remains true even today; how else to explain, as the scholar Jytte Klausen noted in a 2009 paper, the depiction of the Imam Ali and members of the prophet Muhammad's family in "the posters and wall hangings for sale in the bazaars of Tehran and Istanbul"?

For hundreds of years, the Buddhas of Bamiyan managed to be simultaneously featureless and eloquent; their faces had been rendered blank. "No statue which has had its face removed can express justice or law or illumination or mercy," the poet Peter Levi wrote in 1970, "but there is a disturbing presence about these two giants that does express something." The Buddhas continue still to communicate presence through absence, their empty niches in Bamiyan's ochre cliffs speaking to mankind's distressing tendency to discard the very qualities Levi cited: justice, law, illumination and mercy.

• Samanth Subramanian's Following Fish: Travels around the Indian Coast, will be published by Atlantic this summer.