Like many Englishmen who suffered from tuberculosis in the 19th century, Sir Edward Burnett Tylor went abroad on medical advice, seeking the drier air of warmer regions. Tylor came from a prosperous Quaker business family, so he had the resources for a long trip. In 1855, in his early 20s, he left for the New World, and, after befriending a Quaker archeologist he met on his travels, he ended up riding on horseback through the Mexican countryside, visiting Aztec ruins and dusty pueblos. Tylor was impressed by what he called “the evidence of an immense ancient population”. And his Mexican sojourn fired in him an enthusiasm for the study of faraway societies, ancient and modern, that lasted for the rest of his life. In 1871, he published his masterwork, Primitive Culture, which can lay claim to being the first work of modern anthropology.

Primitive Culture was, in some respects, a quarrel with another book that had “culture” in the title: Matthew Arnold’s Culture and Anarchy, a collection that had appeared just two years earlier. For Arnold, culture was the “pursuit of our total perfection by means of getting to know, on all the matters which most concern us, the best which has been thought and said in the world”. Arnold wasn’t interested in anything as narrow as class-bound connoisseurship: he had in mind a moral and aesthetic ideal, which found expression in art and literature and music and philosophy.

But Tylor thought that the word could mean something quite different, and in part for institutional reasons, he was able to see that it did. For Tylor was eventually appointed to direct the University Museum at Oxford, and then, in 1896, he was appointed to the first chair of anthropology there. It is to Tylor more than anyone else that we owe the idea that anthropology is the study of something called “culture”, which he defined as “that complex whole which includes knowledge, belief, arts, morals, law, customs, and any other capabilities and habits acquired by man as a member of society”. Civilisation, as Arnold understood it, was merely one of culture’s many modes.

Nowadays, when people speak about culture, it is usually either Tylor’s or Arnold’s notion that they have in mind. The two concepts of culture are, in some respects, antagonistic. Arnold’s ideal was “the man of culture” and he would have considered “primitive culture” an oxymoron. Tylor thought it absurd to propose that a person could lack culture. Yet these contrasting notions of culture are locked together in our concept of western culture, which many people think defines the identity of modern western people. So let me try to untangle some of our confusions about the culture, both Tylorian and Arnoldian, of what we have come to call the west.

Someone asked Mahatma Gandhi what he thought of western civilisation, and he replied: “I think it would be a very good idea.” Like many of the best stories, alas, this one is probably apocryphal; but also like many of the best stories, it has survived because it has the flavour of truth. But my own response would have been very different: I think you should give up the very idea of western civilisation. It is at best the source of a great deal of confusion, at worst an obstacle to facing some of the great political challenges of our time. I hesitate to disagree with even the Gandhi of legend, but I believe western civilisation is not at all a good idea, and western culture is no improvement.

One reason for the confusions “western culture” spawns comes from confusions about the west. We have used the expression “the west” to do very different jobs. Rudyard Kipling, England’s poet of empire, wrote, “Oh, east is east and west is west, and never the twain shall meet”, contrasting Europe and Asia, but ignoring everywhere else. During the cold war, “the west” was one side of the iron curtain; “the east” its opposite and enemy. This usage, too, effectively disregarded most of the world. Often, in recent years, “the west” means the north Atlantic: Europe and her former colonies in North America. The opposite here is a non-western world in Africa, Asia and Latin America – now dubbed “the global south” – though many people in Latin America will claim a western inheritance, too. This way of talking notices the whole world, but lumps a whole lot of extremely different societies together, while delicately carving around Australians and New Zealanders and white South Africans, so that “western” here can look simply like a euphemism for white.

Of course, we often also talk today of the western world to contrast it not with the south but with the Muslim world. And Muslim thinkers sometimes speak in a parallel way, distinguishing between Dar al-Islam, the home of Islam, and Dar al-Kufr, the home of unbelief. I would like to explore this opposition further. Because European and American debates today about whether western culture is fundamentally Christian inherit a genealogy in which Christendom is replaced by Europe and then by the idea of the west.

This civilisational identity has roots going back nearly 1,300 years, then. But to tell the full story, we need to begin even earlier.

For the Greek historian Herodotus, writing in the fifth century BC, the world was divided into three parts. To the east was Asia, to the south was a continent he called Libya, and the rest was Europe. He knew that people and goods and ideas could travel easily between the continents: he himself travelled up the Nile as far as Aswan, and on both sides of the Hellespont, the traditional boundary between Europe and Asia. Herodotus admitted to being puzzled, in fact, as to “why the earth, which is one, has three names, all women’s”. Still, despite his puzzlement, these continents were for the Greeks and their Roman heirs the largest significant geographical divisions of the world.

But here’s the important point: it would not have occurred to Herodotus to think that these three names corresponded to three kinds of people: Europeans, Asians, and Africans. He was born at Halicarnasus – Bodrum in modern Turkey. Yet being born in Asia Minor didn’t make him an Asian; it left him a Greek. And the Celts, in the far west of Europe, were much stranger to him than the Persians or the Egyptians, about whom he knew rather a lot. Herodotus only uses the word “European” as an adjective, never as a noun. For a millennium after his day, no one else spoke of Europeans as a people, either.

Then the geography Herodotus knew was radically reshaped by the rise of Islam, which burst out of Arabia in the seventh century, spreading with astonishing rapidity north and east and west. After the prophet’s death in 632, the Arabs managed in a mere 30 years to defeat the Persian empire that reached through central Asia as far as India, and to wrest provinces from Rome’s residue in Byzantium.

The Umayyad dynasty, which began in 661, pushed on west into north Africa and east into central Asia. In early 711, it sent an army across the straits of Gibraltar into Spain, which the Arabs called al-Andalus, where it attacked the Visigoths who had ruled much of the Roman province of Hispania for two centuries. Within seven years, most of the Iberian Peninsula was under Muslim rule; not until 1492, nearly 800 years later, was the whole peninsula under Christian sovereignty again.

The Muslim conquerors of Spain had not planned to stop at the Pyrenees, and they made regular attempts in the early years to move further north. But near Tours, in 732CE, Charles Martel, Charlemagne’s grandfather, defeated the forces of al-Andalus, and this decisive battle effectively ended the Arab attempts at the conquest of Frankish Europe. The 18th-century historian Edward Gibbon, overstating somewhat, observed that if the Arabs had won at Tours, they could have sailed up the Thames. “Perhaps,” he added, “the interpretation of the Koran would now be taught in the schools of Oxford, and her pulpits might demonstrate to a circumcised people the sanctity and truth of the revelation of Mahomet.”

The world according to Herodotus. Photograph: Interfoto/Alamy/Alamy

What matters for our purposes is that the first recorded use of a word for Europeans as a kind of person, so far as I know, comes out of this history of conflict. In a Latin chronicle, written in 754 in Spain, the author refers to the victors of the Battle of Tours as “Europenses”, Europeans. So, simply put, the very idea of a “European” was first used to contrast Christians and Muslims. (Even this, however, is a bit of a simplification. In the middle of the eighth century much of Europe was not yet Christian.)

Now, nobody in medieval Europe would have used the word “western” for that job. For one thing, the coast of Morocco, home of the Moors, stretches west of Ireland. For another, there were Muslim rulers in the Iberian Peninsula – part of the continent that Herodotus called Europe – until nearly the 16th century. The natural contrast was not between Islam and the west, but between Christendom and Dar al‑Islam, each of which regarded the other as infidels, defined by their unbelief.

Starting in the late 14th century, the Turks who created the Ottoman empire gradually extended their rule into parts of Europe: Bulgaria, Greece, the Balkans, and Hungary. Only in 1529, with the defeat of Suleiman the Magnificent’s army at Vienna, did the reconquest of eastern Europe begin. It was a slow process. It wasn’t until 1699 that the Ottomans finally lost their Hungarian possessions; Greece became independent only in the early 19th century, Bulgaria even later.

We have, then, a clear sense of Christian Europe – Christendom – defining itself through opposition. And yet the move from “Christendom” to “western culture” isn’t straightforward.

For one thing, the educated classes of Christian Europe took many of their ideas from the pagan societies that preceded them. At the end of the 12th century, Chrétien de Troyes, born a couple of hundred kilometres south-west of Paris, celebrated these earlier roots: “Greece once had the greatest reputation for chivalry and learning,” he wrote. “Then chivalry went to Rome, and so did all of learning, which now has come to France.”

The idea that the best of the culture of Greece was passed by way of Rome into western Europe gradually became, in the middle ages, a commonplace. In fact this process had a name. It was called the “translatio studii”: the transfer of learning. And it was an astonishingly persistent idea. More than six centuries later, Georg Wilhelm Friedrich Hegel, the great German philosopher, told the students of the high school he ran in Nuremberg: “The foundation of higher study must be and remain Greek literature in the first place, Roman in the second.”

So from the late middle ages until now, people have thought of the best in the culture of Greece and Rome as a civilisational inheritance, passed on like a precious golden nugget, dug out of the earth by the Greeks, transferred, when the Roman empire conquered them, to Rome. Partitioned between the Flemish and Florentine courts and the Venetian Republic in the Renaissance, its fragments passed through cities such as Avignon, Paris, Amsterdam, Weimar, Edinburgh and London, and were finally reunited – pieced together like the broken shards of a Grecian urn – in the academies of Europe and the United States.

There are many ways of embellishing the story of the golden nugget. But they all face a historical difficulty; if, that is, you want to make the golden nugget the core of a civilisation opposed to Islam. Because the classical inheritance it identifies was shared with Muslim learning. In Baghdad of the ninth century Abbasid caliphate, the palace library featured the works of Plato and Aristotle, Pythagoras and Euclid, translated into Arabic. In the centuries that Petrarch called the Dark Ages, when Christian Europe made little contribution to the study of Greek classical philosophy, and many of the texts were lost, these works were preserved by Muslim scholars. Much of our modern understanding of classical philosophy among the ancient Greeks we have only because those texts were recovered by European scholars in the Renaissance from the Arabs.

In the mind of its Christian chronicler, as we saw, the battle of Tours pitted Europeans against Islam; but the Muslims of al-Andalus, bellicose as they were, did not think that fighting for territory meant that you could not share ideas. By the end of the first millennium, the cities of the Caliphate of Cordoba were marked by the cohabitation of Jews, Christians, and Muslims, of Berbers, Visigoths, Slavs and countless others.

There were no recognised rabbis or Muslim scholars at the court of Charlemagne; in the cities of al-Andalus there were bishops and synagogues. Racemondo, Catholic bishop of Elvira, was Cordoba’s ambassador to the courts of the Byzantine and the Holy Roman empires. Hasdai ibn Shaprut, leader of Cordoba’s Jewish community in the middle of the 10th century, was not only a great medical scholar, he was the chairman of the Caliph’s medical council; and when the Emperor Constantine in Byzantium sent the Caliph a copy of Dioscorides’s De Materia Medica, he took up Ibn Shaprut’s suggestion to have it translated into Arabic, and Cordoba became one of the great centres of medical knowledge in Europe. The translation into Latin of the works of Ibn Rushd, born in Cordoba in the 12th century, began the European rediscovery of Aristotle. He was known in Latin as Averroes, or more commonly just as “The Commentator”, because of his commentaries on Aristotle. So the classical traditions that are meant to distinguish western civilisation from the inheritors of the caliphates are actually a point of kinship with them.

The term 'western culture' is surprisingly modern – more recent certainly than the phonograph

But the golden-nugget story was bound to be beset by difficulties. It imagines western culture as the expression of an essence – a something – which has been passed from hand to hand on its historic journey. The pitfalls of this sort of essentialism are evident in a wide range of cases. Whether you are discussing religion, nationality, race or culture, people have supposed that an identity that survives through time and space must be propelled by some potent common essence. But that is simply a mistake. What was England like in the days of Chaucer, father of English literature, who died more than 600 years ago? Take whatever you think was distinctive of it, whatever combination of customs, ideas, and material things that made England characteristically English then. Whatever you choose to distinguish Englishness now, it isn’t going to be that. Rather, as time rolls on, each generation inherits the label from an earlier one; and, in each generation, the label comes with a legacy. But as the legacies are lost or exchanged for other treasures, the label keeps moving on. And so, when some of those in one generation move from the territory to which English identity was once tied – move, for example, to a New England – the label can even travel beyond the territory. Identities can be held together by narratives, in short, without essences. You don’t get to be called “English” because there’s an essence that this label follows; you’re English because our rules determine that you are entitled to the label by being somehow connected with a place called England.

So how did the people of the north Atlantic, and some of their kin around the world, get connected to a realm we call the west, and gain an identity as participants in something called western culture?

James Gillray’s 1805 cartoon, The Plumb Pudding in Danger, depicts prime minister William Pitt and Napoleon Bonaparte carving up the world Photograph: Rischgitz/Getty Images

It will help to recognise that the term “western culture” is surprisingly modern – more recent certainly than the phonograph. Tylor never spoke of it. And indeed he had no reason to, since he was profoundly aware of the internal cultural diversity even of his own country. In 1871 he reported evidence of witchcraft in rural Somerset. A blast of wind in a pub had blown some roasted onions stabbed with pins out of the chimney. “One,” Tylor wrote, “had on it the name of a brother magistrate of mine, whom the wizard, who was the alehouse-keeper, held in particular hatred ... and whom apparently he designed to get rid of by stabbing and roasting an onion representing him.” Primitive culture, indeed.

So the very idea of the “west,” to name a heritage and object of study, doesn’t really emerge until the 1890s, during a heated era of imperialism, and gains broader currency only in the 20th century. When, around the time of the first world war, Oswald Spengler wrote the influential book translated as The Decline of the West – a book that introduced many readers to the concept – he scoffed at the notion that there were continuities between western culture and the classical world. During a visit to the Balkans in the late 1930s, the writer and journalist Rebecca West recounted a visitor’s sense that “it’s uncomfortably recent, the blow that would have smashed the whole of our western culture”. The “recent blow” in question was the Turkish siege of Vienna in 1683.

To be blunt: if western culture were real, we wouldn’t spend so much time talking it up

If the notion of Christendom was an artefact of a prolonged military struggle against Muslim forces, our modern concept of western culture largely took its present shape during the cold war. In the chill of battle, we forged a grand narrative about Athenian democracy, the Magna Carta, Copernican revolution, and so on. Plato to Nato. Western culture was, at its core, individualistic and democratic and liberty-minded and tolerant and progressive and rational and scientific. Never mind that pre-modern Europe was none of these things, and that until the past century democracy was the exception in Europe – something that few stalwarts of western thought had anything good to say about. The idea that tolerance was constitutive of something called western culture would have surprised Edward Burnett Tylor, who, as a Quaker, had been barred from attending England’s great universities. To be blunt: if western culture were real, we wouldn’t spend so much time talking it up.

Of course, once western culture could be a term of praise, it was bound to become a term of dispraise, too. Critics of western culture, producing a photonegative emphasising slavery, subjugation, racism, militarism, and genocide, were committed to the very same essentialism, even if they see a nugget not of gold but of arsenic.

Talk of “western culture” has had a larger implausibility to overcome. It places, at the heart of identity, all manner of exalted intellectual and artistic achievements – philosophy, literature, art, music; the things Arnold prized and humanists study. But if western culture was there in Troyes in the late 12th century when Chrétien was alive, it had little to do with the lives of most of his fellow citizens, who did not know Latin or Greek, and had never heard of Plato. Today the classical heritage plays no greater role in the everyday lives of most Americans or Britons. Are these Arnoldian achievements that hold us together? Of course not. What holds us together, surely, is Tylor’s broad sense of culture: our customs of dress and greeting, the habits of behaviour that shape relations between men and women, parents and children, cops and civilians, shop assistants and consumers. Intellectuals like me have a tendency to suppose that the things we care about are the most important things. I don’t say they don’t matter. But they matter less than the story of the golden nugget suggests.

So how have we bridged the chasm here? How have we managed to tell ourselves that we are rightful inheritors of Plato, Aquinas, and Kant, when the stuff of our existence is more Beyoncé and Burger King? Well, by fusing the Tylorian picture and the Arnoldian one, the realm of the everyday and the realm of the ideal. And the key to this was something that was already present in Tylor’s work. Remember his famous definition: it began with culture as “that complex whole”. What you’re hearing is something we can call organicism. A vision of culture not as a loose assemblage of disparate fragments but as an organic unity, each component, like the organs in a body, carefully adapted to occupy a particular place, each part essential to the functioning of the whole. The Eurovision song contest, the cutouts of Matisse, the dialogues of Plato are all parts of a larger whole. As such, each is a holding in your cultural library, so to speak, even if you have never personally checked it out. Even if it isn’t your jam, it is still your heritage and possession. Organicism explained how our everyday selves could be dusted with gold.

Britons once swapped their fish and chips for chicken tikka masala, now, I gather, they’re all having a cheeky Nando’s

Now, there are organic wholes in our cultural life: the music, the words, the set-design, the dance of an opera fit and are meant to fit together. It is, in the word Wagner invented, a Gesamtkunstwerk, a total work of art. But there isn’t one great big whole called culture that organically unites all these parts. Spain, in the heart of “the west,” resisted liberal democracy for two generations after it took off in India and Japan in “the east,” the home of Oriental despotism. Jefferson’s cultural inheritance – Athenian liberty, Anglo-Saxon freedom – did not preserve the United States from creating a slave republic. At the same time, Franz Kafka and Miles Davis can live together as easily – perhaps even more easily – than Kafka and his fellow Austro-Hungarian Johann Strauss. You will find hip-hop in the streets of Tokyo. The same is true in cuisine: Britons once swapped their fish and chips for chicken tikka masala, now, I gather, they’re all having a cheeky Nando’s.

Once we abandon organicism, we can take up the more cosmopolitan picture in which every element of culture, from philosophy or cuisine to the style of bodily movement, is separable in principle from all the others – you really can walk and talk like an African-American and think with Matthew Arnold and Immanuel Kant, as well as with Martin Luther King and Miles Davis. No Muslim essence stops the inhabitants of Dar al-Islam from taking up anything from western civilisation, including Christianity or democracy. No western essence is there to stop a New Yorker of any ancestry taking up Islam.

The stories we tell that connect Plato or Aristotle or Cicero or Saint Augustine to contemporary culture in the north Atlantic world have some truth in them, of course. We have self-conscious traditions of scholarship and argumentation. The delusion is to think that it suffices that we have access to these values, as if they are tracks on a Spotify playlist we have never quite listened to. If these thinkers are part of our Arnoldian culture, there is no guarantee that what is best in them will continue to mean something to the children of those who now look back to them, any more than the centrality of Aristotle to Muslim thought for hundreds of years guarantees him an important place in modern Muslim cultures.

Values aren’t a birthright: you need to keep caring about them. Living in the west, however you define it, being western, provides no guarantee that you will care about western civilisation. The values European humanists like to espouse belong just as easily to an African or an Asian who takes them up with enthusiasm as to a European. By that very logic, of course, they do not belong to a European who has not taken the trouble to understand and absorb them. The same, of course, is true in the other direction. The story of the golden nugget suggests that we cannot help caring about the traditions of “the west” because they are ours: in fact, the opposite is true. They are only ours if we care about them. A culture of liberty, tolerance, and rational inquiry: that would be a good idea. But these values represent choices to make, not tracks laid down by a western destiny.

In the year of Edward Burnett Tylor’s death, what we have been taught to call western civilisation stumbled into a death match with itself: the Allies and the Great Central Powers hurled bodies at each other, marching young men to their deaths in order to “defend civilisation”. The blood-soaked fields and gas-poisoned trenches would have shocked Tylor’s evolutionist, progressivist hopes, and confirmed Arnold’s worst fears about what civilisation really meant. Arnold and Tylor would have agreed, at least, on this: culture isn’t a box to check on the questionnaire of humanity; it is a process you join, a life lived with others.

Culture – like religion and nation and race – provides a source of identity for contemporary human beings. And, like all three, it can become a form of confinement, conceptual mistakes underwriting moral ones. Yet all of them can also give contours to our freedom. Social identities connect the small scale where we live our lives alongside our kith and kin with larger movements, causes, and concerns. They can make a wider world intelligible, alive, and urgent. They can expand our horizons to communities larger than the ones we personally inhabit. But our lives must make sense, too, at the largest of all scales. We live in an era in which our actions, in the realm of ideology as in the realm of technology, increasingly have global effects. When it comes to the compass of our concern and compassion, humanity as a whole is not too broad a horizon.

We live with seven billion fellow humans on a small, warming planet. The cosmopolitan impulse that draws on our common humanity is no longer a luxury; it has become a necessity. And in encapsulating that creed I can draw on a frequent presence in courses in western civilisation, because I don’t think I can improve on the formulation of the dramatist Terence: a former slave from Roman Africa, a Latin interpreter of Greek comedies, a writer from classical Europe who called himself Terence the African. He once wrote, “Homo sum, humani nihil a me alienum puto.” “I am human, I think nothing human alien to me.” Now there’s an identity worth holding on to.

• This is an edited version of Kwame Anthony Appiah’s BBC Reith lecture, Culture, the fourth part of the series Mistaken Identities, which is available on the Radio 4 website

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.