My whole life has been riddled with conflict. I’m caught up in a war not of my own making, and must be combat-ready at any time. Or so it seems, to my disquiet, whenever I hear about my immune system: my days appear placid, yet under the surface my body seethes with threats and counter-measures.

This militarisation of biology troubles me. When my children were young, I worried about their immune systems: had they had the right jabs? Might their later teenage indifference to sleep or sensible food have undermined their resistance to, well, I don’t know, life? When my elderly parents were in and out of hospital, I worried that the MRSA ‘superbug’ would get them. In the end, each succumbed to old age. But both death certificates cite bronchial pneumonia – a cause of death often associated with being immunocompromised.

More positively, not to say smugly, I always find it hard not to credit my apparent inability to catch the flu to my having a slightly superior immune system. But that’s a luxury of a healthy life. If I’d grappled with one of a growing list of diseases – not just the obviously infectious ones, but everything from cancer and heart disease to irritable bowel syndrome and arthritis – then my hopes and fears about immune defence would intensify.

And I have another worry: is this actually a good way to understand my body and what it is doing? The imagery of war is hard to get away from. The internet throws up instant reminders. An educational YouTube video posted a couple of years ago by the Norwegian pharmaceutical group Immitec shows a large living cell, seen in time-lapse photos down a microscope, sidling up to a bunch of small, dark blobs and swallowing them up. The big, mean-looking one, a white cell, is doing its duty. As the voiceover has it, it is there to ‘hunt and kill bacteria’.

This idea that there are hunter-killers in my blood and tissues seems at first to help make sense of my otherwise elusive inner world. I know it is populated by billions of cells – far bigger than molecules but still invisible to the unaided eye. Yet I have few intuitions about how they behave. However, if medicine is a war against disease, then perhaps immune cells are a first line of defence.

That is what we teach children. A widely read 1990s book for kids purports to tell ‘the true story of the amazing defenders inside your body, a brave band of cells that keep you healthy by constantly battling against all kinds of invading germs. Every minute, every hour, every day of your life, they are fighting.’ Its title? Cell Wars.

The relentless militarism is wearisome. But alternatives are hard to come by. I seem stuck with these ideas about immunity when I wonder about how healthy I am, or how to stay well. I know that it’s good to have lots of antibodies, stuck on special cells, or circulating in the blood. These are ‘good guys’, protecting me from a whole collection of bad things that can happen to a body. But it feels like a fiction that conceals as much as it reveals. The inner life of immunity, what controls it, how it reacts, are hidden. Some apparatus is busily at work, responding to internal and external nudging, but the effects are rarely manifest. A patch of inflamed skin or a bout of wheeziness after cuddling the neighbour’s cat show themselves, almost like a neurotic tick. But the serious business of the immune system is, like the unconscious, inaccessible to introspection.

So I am still looking for a more helpful way to conceptualise what is going on here than picturing endless combat. Meanwhile, I want to know how the notion that immunity is all to do with threat, invasion, surveillance and defence has maintained its grip – on scientists as much as popular understanding. Other metaphors have been proposed to frame our thinking of immunity, so why have they made so little impression on the idea that is part of arming us for a war against disease?

The metaphor of immune defence – and the linked ideas that make up what has been dubbed ‘biomilitarism’ – has fairly clear origins. Every schoolchild learns about the English scientist Edward Jenner pioneering smallpox vaccinations in 1796, improving on the older practice of variolation. A century later, Louis Pasteur injected a weakened form of cholera into chickens which, stunningly, then survived a lethal dose of the intact organism. This was when the term immunity, which was introduced into medicine from legal usage as early as the 14th century, strengthened its sense of resistance to disease.

Pasteur ­– one of the main architects of germ theory – was a French intellectual who felt humiliated by losing the Franco-Prussian war. Perhaps that drew him to military metaphors, with germs as the enemy; although a Western culture whose literature constantly reaches back to blood-soaked narratives such as the Iliad has little need of more recent conflicts to instill martial thoughts.

In any case, the more direct origin of cell wars came a few years further on. For all his military fixation, in Pasteur’s mind immunity was passive. For him, repeated exposures to bacteria were less likely to lead to lasting infections because essential bacterial nutrients became depleted. Active defence of the organism would be conceived quite differently.

There was real action to examine, as the Russian zoologist Élie Metchnikoff showed in the 1880s. As Metchnikoff related in his Nobel lecture in 1908, he was working on development when he noticed unusual amoeba-like cells that moved around in the transparent larvae of starfish. What, he wondered, were they doing? He took thorns from a decorative tangerine tree, poked them into a few larvae, and left the injured creatures overnight. In the morning, the mobile cells had congregated round the intrusions, like onlookers at a car crash.

the Second World War added to the metaphor of immune defence against invasion

Starfish and tree thorns normally never meet. But perhaps the cells were there to respond to other kinds of insult. Metchnikoff went on to show that the cells – christened phagocytes – engulfed smaller bacterial cells when they entered the blood. The image of cell eating cell is still compelling, as that YouTube video shows.

It was the image that won out.

Metchnikoff himself was an embryologist at heart, and put his mobile cells at the centre of an elaborate theory in which competing cell lineages were fashioned into a unitary organism. Phagocytes animated the process, he suggested, eating up cell lines that were no longer wanted, leaving others to flourish. It was a much broader idea than immune defence. But as the US historian of science Alfred Tauber notes, it was misunderstood or ignored. Fighting germs was the thing, whether the agents were cells or, as most believed, special molecules in the circulation. As Tauber wrote in the Immune Self (1997): ‘Metchnikoff’s concept of immunity… was generally understood solely as defensive action against pathogenic agencies.’

That understanding underpinned amply productive research. In the first half of the 20th century, researchers mapped immune responses and exploited them in new mass vaccinations. Scientific work was still mostly watching cells under the microscope, and recording what stuck to what, or ate what, and what reproduced or died. Such observations led to a proliferation of functional molecules that theoretically ought to exist. More and more were given descriptive names – antitoxins, precipitins, bacteriolysins, agglutinins, opsonins, reagins, conglutinins – derived from their apparent effects. It was a first taste of a feeling that 21st-century immunology easily evokes. As the late, great British evolutionist John Maynard Smith once told me: ‘The problem with biology is that there are too many damn facts.’

By the 1930s, mercifully, this jumble of entities was unified. They were, it was agreed, all varieties of the same thing, what the German physician Paul Ehrlich decades earlier had dubbed antibodies. Two research problems then dominated the next decades. If antibodies were the agents that reacted to invasion, it wasn’t clear how they could recognise foreign stuff so specifically or respond to so many different challenges. Neither question, though, threatened the idea that defence was the key to understanding immunity.

Most proposals about what some scientists dubbed the ‘generator of diversity’ (or GOD) were template theories – imagining how an intruding antigen somehow directed formation of the complementary antibody. While experimenters struggled to test these theories, more details emerged of the way in which antibodies stimulate other immune cells. Eventually, these new understandings came together with experiments on tissue transplantation, partly inspired by work with burns victims in the Second World War, adding to the metaphor of immune defence against invasion the new (also metaphorical) notion of ‘tolerance’ of antigens that did not provoke an immune reaction.

Work on transplantation and tolerance helped to prompt another fresh metaphor, as the modern notion of antibody specificity – originating with the Australian virologist Frank Macfarlane Burnet and the Danish immunologist Niels Kaj Jerne – took hold after 1945. Burnet and Jerne proposed that the key to the tolerance puzzle – which led to the destruction of perfectly functional donor tissue while our own organs remain unscathed – is self/non-self recognition. This dovetailed with a new alternative to template theories of antibody design – a random generation and selection theory that brought a variant of Darwinian thinking to immune development inside the organism.

Here’s their story. We first make immune cells that recognise antigens. Not just some antigens, like those that might denote something nasty, but a whole universe, consisting of trillions of molecular shapes. Immune cells that respond to our own antigens are then eliminated before they reach the general circulation. The others remain, including some of the cells that make one specific antibody reproduce explosively when they meet the right ‘non-self’ antigen. The experiments that firmed up this theory went on until the early 1970s, when it was recognised with a clutch of Nobel prizes.

Self versus non-self conjures up a regime for stopping ‘foreign’ entities at the borders, in this case of the body

By this time, we also knew the source of variation among antibodies. A special set of genes shuffle stretches of their DNA, creating many different antibody proteins. Uniquely, this reshuffling does not happen in reproductive cells, but in bone marrow cells. This made sense as bacteria and viruses evolve very fast. If responding to their mutations depended on changes in the human germ-line, with a generation time of a few tens of years, the mismatch would be conspicuous.

This new framework, with its molecular and genetic insights, was underpinned by the Burnet/Jerne theory that immunity depended on distinguishing between self and non-self. The organism developed a record of antigens it had encountered – whether they were generated internally or externally. But the record varied for self antigens, which did not demand an immune response and were marked as safe, and non-self antigens that originated elsewhere. The latter were remembered after a first encounter as threats that might return, when surgical strikes were in order.

One reason it has been hard to shake military and defence metaphors becomes clearer. They are easy to elaborate and can accommodate many kinds of complexity. Self versus non-self conjures up a regime for stopping ‘foreign’ entities at the borders, in this case of the body. Germs remained the enemy: the defence now focused on identifying them. It depended on immune surveillance and response – or military intelligence. The price of good health was eternal vigilance by agents patrolling the boundaries of the self. And autoimmune diseases? Friendly fire, obviously.

The next modern idea – of the immune system – arose after the apparatus supporting immunity revealed yet more complexity and intricacies of cellular interaction. As Burnet once said, it is ‘an intensely biological invention’. The catalogue of its components included not just super-variable antibodies, but many different classes of antibody molecules, associated with different kinds of cells. And biologists teased out the roles of many other molecules that continually pass between cells, or trip switches inside them.

The long list of important agents made earlier immune inventories look simple. There were new kinds of cells, with crowds of sub-types. There were a host of molecules, most famously antibodies but not only those, on the surface of all these cells. And there were yet others in the circulation, many of them smaller signalling molecules, activating, suppressing, modulating and modifying everything the cells do. All this was summed up in the metaphor of an immune system. This was somewhat vague, expressing the idea that everything was connected to everything else, rather than anything more precise from systems theory.

The connections were partly between scattered sites in the body. Immune cells originate in the bone marrow, but also pass through those obscure organs the thymus and spleen, whose roles were elucidated after key observations about lymphocyte circulation by the British immunologist James Learmonth Gowans in the late 1950s. But the system is not located anywhere in particular. It must be everywhere to do its job. If the immune system were a single organ, it would be a pretty large one. But it is not. It is diffuse, protean, multi-faceted. Immune cells move all around our bodies. There are immune molecules in blood, sweat and tears.

‘System’ was a biologists’ way of insisting that all these widely distributed components were part of the same thing, and that command and control of immune responses has to be coordinated. Hence all those signalling molecules. Immunologists also hoped that all their experiments related to a definable entity. Nobody worked on the immune system as such. Rather, thousands of scientists worked on hundreds of different set-ups – using standardised mice, or just assemblies of cells, molecules and instruments in the lab – and then argued about where they fit into a larger picture. Burnet reached for another, science-inspired, metaphor: even the good experiments, he said, were no better than two-dimensional sections of a multidimensional immune universe.

In the public realm, considering immunity as something conferred by an immune system made little difference. It blended seamlessly with ill-defined ideas about health maintenance. Toning up the immune system could help to defeat the nasties. Letting it run down was part of being run down. Defence was still the basic agenda, and there were plenty of products in the health food store to help keep your defences up.

More troublingly for anti-militarists was a darker turn in public discussion of immunity in the 1980s with the advent of AIDS. The public prominence of immunity, and worries about how it might be weakened, were then boosted by the new epidemic, described as a disease of the immune system. Bits of the technical vocabulary of immunology gained wider currency. Everyone suddenly seemed to know what killer T cells were. Now the metaphors of war, surveillance, codes, combat and infiltration came together in ugly new combinations. Healthy, right-thinking folk trained their immune systems to behave properly. But the reputed sources of HIV infection – drug injection, ‘unnatural’ sexual practices, immigrants – were all portrayed as breaches of the boundaries that the immune system was supposed to defend. The consequent immunocompromising infection, in other words, was self-inflicted. The immunology of self and non-self was recruited to the perennial effort to blame the victim.

The US anthropologist Emily Martin was among those made uneasy by this. In her book Flexible Bodies (1994), she wondered if there were alternatives to the idea of the body at war, but she found it still dominated professional discussions of immunity and its popular depictions. Even those consciously trying to get away from biomilitarism found it hard to come up with alternatives. In the end, one scientist told her, he saw the immune system in these terms but, ‘this is a just war’.

A few new metaphors did crop up here and there. Researchers introduced ideas from cognition and information theory into immunology talk. The immune system must have a dictionary, Niels Jerne once wrote, which it consults to decide whether such and such a chemical word is foreign or belongs to its own language. Popular writers tried other metaphors, most often musical ones – the immune orchestra, the immune symphony. These are evocative, but hardly the basis for a research programme.

in computer security, that metaphorical but annoyingly oh-so-real entity, the computer virus, is often the problem

Other ideas sounded more like worked-through scientific proposals. One, described by Jerne in the 1970s, was a variant of the systems view: he advocated seeing the whole set of immune cells, antibody and antigen binding sites, and the numerous chemical modulators, as a self-regulating network. The immune network, Jerne believed, had emergent properties that put it on a level with the nervous system. This immune network, which grants an important role to the head-achingly self-referential proteins that are antibodies to antibody molecules (and which can generate their own antibodies in turn), has cognition, of a kind. It can represent things to itself, both internal and external. It can learn. It can remember. More familiarly, it can react. It seemed more than a simple defence-against-invasion system. It was a network spanning the whole body, with properties to be explored through complex computational modelling as much as through experiments with cells in dishes or whole organisms.

The network notion seemed fashionable for a while, in accord with the vogue for complexity and complex systems that swept science in the 1980s and ’90s. But after a flurry of interest, it had little lasting influence on immunology. Some experimenters found evidence of the kinds of interactions and interconnections that Jerne envisaged, but not enough to convince his colleagues that network properties were the most important feature of the immune system.

On the other hand, these ideas were very congenial to computer scientists and software engineers, and there is a flourishing research community building ‘artificial immune systems’ that exist only inside silicon circuits. Some of this work has real-world uses, in data analysis and pattern recognition. Yet even here, the preoccupation with defence creeps back in. Immune-system-inspired software developers have worked on computer security, where that metaphorical but annoyingly oh-so-real entity, the computer virus, is often the problem.

Meanwhile, another avowedly radical idea, which attracted a good deal of press in the 1990s, was pitched against the notion that recognition of self was the immune system’s main job. Polly Matzinger, a researcher at the US National Institutes of Health, suggested it made more sense to see it, instead, as a set of capacities to respond to injuries of all kinds. This ‘danger’ theory attracted few adherents, with critics dismissing it as tautologous for defining anything that evoked an immune response as a danger signal.

For me, still hoping that the immune system might escape conscription, the distance between the danger theory and defence seemed small. Once again, the military theory can easily incorporate this idea: danger signals are what alerts the defences.

However, one aspect of Matzinger’s thinking pointed to a different, potentially more fruitful metaphor. It anticipated more recent suggestions that could finally offer an escape from biomilitarism. These point away from inbuilt oppositions, from them-and-us, and toward something more communicative, collaborative. As Matzinger put it in a paper in Science in 2002, immunity ‘is controlled by an internal conversation between tissues and the cells of the immune system’.

In the past decade, our sense of that conversation has deepened. In particular, it’s become clear that many of the most important exchanges aren’t with our own cells at all. We constantly talk to bacterial cells that live on and in us, especially the trillions of them in our gut.

This is one of a sequence of new discoveries about the human microbiome – the collection of microorganisms that we carry around with us. We have known that they were there since the 19th century. But their true diversity, and their influence on the rest of the body, remained largely hidden until the advent of the latest technologies for sequencing DNA. It turns out that bacteria – some of them anyway – are not evil. They are our boon companions, helping digest our food, making vitamins, and all the while interacting with us. Those interactions invite us to take a very different view of the immune system.

That view begins with a few key facts. Humans have a large and complex microbiome. Most lies in an incomparably diverse bacterial ecosystem in the gut. The intestines, with an internal surface area roughly the size of a tennis court, are also richly populated with immune cells. These continually sample the contents of the intestine, and register which bacteria dwell there.

That is the news from our insides, much of it from the US National Institutes of Health-coordinated Human Microbiome Project, and a similar European effort. It cuts across the self/non-self distinction, blurring what actually counts as a self. With our microbiome, we now look like a continually remade coalition of cells. Each of us includes a shifting, shimmering population of other organisms. We are not at war with them: their continued presence is essential for our wellbeing.

That’s true for many other creatures, too. But not all. Every complex organism evolved surrounded by microbes, and all have a microbiome of some kind. However, complex communities of bacteria such as ours are found only in vertebrates.

And creatures with backbones, starting with fish, have something else in common. Their adaptive immune system has the full panoply of cells and antibodies. The rest make do with a simpler, innate immune system. This responds to bacteria in general, recognising molecules that bacterial cell surfaces have in common. But it doesn’t distinguish much between them.

Such a system fits in with ideas about defence. But if that works, it is hard to explain why all the other components evolved later on. Unless, as the US zoologist Margaret McFall-Ngai suggested in Nature in 2007: ‘The evolution of the vertebrate immune system is… strongly affected by the need to maintain a substantial resident microbiota.’ For her, the whole point of the adaptive immune system is to help work out whom not to attack.

the immune system is like a bouncer, ‘raising the velvet rope for beneficial bacteria and giving attitude to their less desirable brethren’

Indeed, our immune system does more than leave some beneficial bacteria alone. It even helps encourage the right ones to colonise our insides. The protein immunoglobulin A, for example, recognises surface molecules on some gut bacteria, as a defence aid might. Does that signal their destruction? No. Binding the protein actually helps the bacteria cling to the gut wall, where it grows into a well-established biofilm.

Findings such as this have prompted a flurry of new metaphors from immunologists. McFall-Ngai’s proposal means that the immune system is a bit like a bouncer, says one, ‘raising the velvet rope for beneficial bacteria and giving attitude to their less desirable brethren’.

Others go further. Which bacteria are beneficial, which less desirable, argues Gérard Eberl of the Pasteur Institute in Paris, depends on the context. We have to give up on dualist thinking. No germ is all bad: ‘[M]icrobes navigate between shades of good and evil.’ The immune system keeps host and microbiome in equilibrium. There is continual action and reaction, like the give and take of a regular conversation. The results help to nurture some bacteria, while reducing opportunities for others. The bacteria, meanwhile, help to regulate normal development of their host. Sarkiz Mazmanian of CalTech in Pasadena, for example, has showed that mice won’t develop a normal immune system unless the right bacteria inhabit their bowels in infancy.

It would be too much to claim that all this completely redraws the picture of an internal defence system. Some immune cells remain hard to see as anything but weaponised agents, waiting for marching orders. But superbly elaborate checks and balances control their deployment, as one hopes for real-world weaponry. And much of the adaptive immune system is now discussed more often in terms of conversation, negotiation, cooperation, even diplomacy.

The immune armed forces, in other words, are still there. But in the real world of the body, they are subordinate to other, more subtly evolved influences, concerned with the fine detail of exchanges between the many other species we join with in a larger community of cells. I can live with that.