The classic novel by H G Wells, The War of the Worlds (1898) – a tale of England besieged by Martian conquerors – ends not with a rousing and heroic victory but an accidental one. The aliens subjugate humanity with heat rays and black smoke but, at the height of their victory, they die. Their machines come to a standstill amid the ruins of a deserted London, and the birds pick at their rotting remains. The cause of their downfall? Bacteria. As the novel’s unnamed narrator says, they were ‘slain, after all man’s devices had failed, by the humblest things’.

Wells’s logic was simple. Humans have immune systems that protect us from the infectious germs that we’ve been exposed to since our earliest origins. We still get diseases, but at least we can put up a fight. The Martians, despite their technological superiority, could not. ‘There are no bacteria in Mars,’ the narrator explains, ‘and directly these invaders arrived, directly they drank and fed, our microscopic allies began to work their overthrow.’

When I first read the book about two decades ago, this final twist seemed like a cop-out. It comes out of nowhere – a sort of deus ex microbia rescue – and, besides, Earth’s microbes could not possibly grow in an alien body. But more recently, I have come to realise that Wells, writing at the close of the 19th century, was inadvertently hinting at a truth about bacteria that even today’s microbiologists sometimes forget: these organisms can become lethal through evolutionary accidents.

In the novel, bacteria did not evolve to cripple the aliens. They evolved to target humans and other animals. The invaders unexpectedly stepped into the crossfire, and succumbed. The same can happen to us. Many of the bacteria and fungi that afflict us with severe diseases are not aiming at us at all. Instead, they have evolved to thrive in harsh environments or to fend off other microbes. It just so happens that these same adaptations allow them to thrive in our bodies or to fend off our immune systems.

Consider Streptococcus pneumoniae, a common bacterium that lives in our noses and airways. It is typically harmless but it can sometimes flip from passive passenger to active killer, causing pneumonia, meningitis, sepsis, and other illnesses. This typically happens in people with weakened immune systems, and it’s often the fault of strains that have thick coats of complex sugar molecules that protect them from our immune systems.

But in 2007 Elena Lysenko, then at the University of Pennsylvania School of Medicine, found that there is more to this story than strong microbe meets compromised host. S.pneumoniae shares our airways with other bacteria such as Hemophilus influenzae. These species do not make for happy neighbours, and H.influenzae has been known to marshal its host’s white blood cells to attack its competitor. This strategy usually works. When Lysenko incubated the two microbes together in mice, she found the rodents usually ousted S.pneumoniae from their airways, leaving H.influenzae to rule alone.

But the thickly armoured strains of S.pneumoniae are impervious to white blood cells, and can stand their ground. Their armour would normally be a liability – they take so much energy to make that their owners get outcompeted by strains that make lighter and less costly coats. But with H.influenzae mobilising an immune army, a thick coat suddenly becomes worthwhile. And by coincidence, those coats make these strains better at invading deeper parts of the respiratory system, and causing serious disease. In defending itself from a competitor, S.pneumoniae inadvertently becomes an armoured killer.

Its virulence – its ability to cause disease – is not an adaptation against its host. It is a side effect, a fluke. It kills through coincidence.

In the late 17th century, the Dutch scientist Antonie van Leeuwenhoek created a new type of microscope lens and brought an entire world of tiny organisms into focus. Looking at his own dental plaque, he wrote: ‘I then most always saw with great wonder, that in the said matter there were many very little living animalcules, very prettily a-moving.’ These little creatures were intriguing but seemingly unimportant, and few others picked up the baton from van Leeuwenhoek. That changed in the 19th century, when Louis Pasteur and Robert Koch proved that some of these microbes were behind important diseases.

That framing has stuck. Microbes are everywhere, but we take their presence on phones, keyboards, and toilet seats as a sign of filth and squalor. They fill our bodies, helping us to digest our food and safeguard our health, but we view them as adversaries to be drugged and conquered.

This antagonism is understandable. Aside from those of us with access to microscopes, most people will never see microbes with their own eyes. And so we tend to identify microbes with the disease-causing minority among them, the little buggers that trigger the tickling mist of a sneeze or the pustule on otherwise smooth skin. We become aware of their existence when they threaten our lives, and for much of our history, that threat was substantial. Epidemics of smallpox, cholera, tuberculosis, and plague have traumatised humanity, and the fear of these diseases has contaminated our entire culture, from our religious rites to Hollywood films such as Contagion (2012) or Outbreak (1995).

When microbes aren’t killing us, we are largely oblivious to them. So, we construct narratives of hosts and pathogens, heroes and villains, us and them. Those that cause disease exist to reproduce at our expense, and we need new ways of resisting them. And so we study how they evolve to outfox our immune system or to spread more easily from one person to another. We identify genes that allow them to cause disease and we label those genes as ‘virulence factors’. We place ourselves at the centre of their world. We make it all about us.

We might think of antibiotics as modern inventions, but they’re actually weapons that bacteria have been using against each other for aeons

But a growing number of studies show that our anthropocentric view is sometimes unjustified. The adaptations that allow bacteria, fungi and other pathogens to cause us harm can easily evolve outside the context of human disease. They are part of a microbial narrative that affects us, and can even kill us, but that isn’t about us. This concept is known as the coincidental evolution hypothesis or, as the Emory University microbiologist Bruce Levin described it in 2008, the ‘shit happens’ hypothesis.

This hypothesis does not apply to all infections, and is almost certainly irrelevant to viruses, which always need to reproduce in a host. Nor does it apply to the many bacteria and fungi, such as Staphylococcus aureus or Candida albicans, that are long-standing human pathogens and well-adapted to us. But it does explain some weird aspects of many diseases.

Why, for example, would bacteria harm the hosts that they depend on for survival? In some cases, the answer is obvious: they cause symptoms such as sneezing or coughing that help them to spread. But what about S.pneumoniae? Strains that sit harmlessly in a host’s airways are already capable of spreading to another individual. The virulent forms, which descend deeper into the respiratory tract, are actually less contagious. The same goes for bugs such as Hemophilus influenzae and Neisseria meningitidis, which can inflame the protective membranes around the brain and lead to life-threatening cases of bacterial meningitis. In doing so, they risk capsizing their own ship without any hope of boarding a new one.

The coincidental evolution hypothesis helps to resolve these paradoxes. It tells us that at least some human diseases have nothing to do with us at all.

The coincidental evolution hypothesis explains a number of other recent discoveries about microbes. Scientists have found antibiotic resistance genes in bacteria that have been frozen for 30,000 years, or isolated in million-year-old caves. We might think of antibiotics as modern inventions, but they’re actually weapons that bacteria have been using against each other for aeons, or at least well before Alexander Fleming noticed a funky mould in a Petri dish in 1928. Antibiotic resistance genes evolved as part of this ancient war, but they also help today’s microbes to deal with the medicines that we mass-produce.

Likewise, many of the ‘virulence genes’ that help pathogens to cause disease have counterparts in marine microbes with no track record of infecting humans. And some supposedly pathogenic bacteria were often common parts of the environment. ‘These organisms become accidental pathogens,’ says the microbiologist Arturo Casadevall from Yeshiva University in New York. ‘They’ll still be there even if you remove all the animals from the planet. And yet, evolution selected for just the right combination of traits to cause disease in humans.’

We fear lions and tigers and bears; bacteria have to contend with phage viruses, nematode worms, and predatory amoebas

Vibrio cholerae, the bacterium that causes cholera, is a good example. Scientists used to regard it as a human pathogen that spreads when the faeces of infected people seep into water supplies. We now know that it’s mainly a marine species that attaches itself to the shells of small crustaceans, and occasionally makes its way into our water supply. ‘In the last decade, people have begun to accept that a lot of these opportunistic pathogens that people assumed were only in the environment transiently between human hosts are actually environmental bacteria that occasionally end up in humans,’ says Diane McDougald from the University of New South Wales, who studies V.cholerae.

Many of the pathogens we fear most are mere tourists on the human body. Their real homes are oceans, caves, or soils. To understand them, we need to understand them within their natural ecology. Soil, for example, is an extreme habitat for a microbe: harsh and constantly changing. It can quickly oscillate from flood to drought, from scalding heat to freezing cold, and total darkness to intense solar radiation. It’s rife with other competing microbes, and crawling with hungry predators. We fear lions and tigers and bears; bacteria have to contend with phage viruses, nematode worms, and predatory amoebas.

All of these conditions can lead to adaptations that make microbes accidentally suited for life in a human host. We are, after all, just another environment. A thick capsule that shields a microbe from dehydration could also shield it from our immune system. A spore that is adapted for travelling through the air can be easily inhaled into a respiratory tract.

Scientists have demonstrated many of these coincidental adaptations by exposing bacteria to a natural threat and showing that they then become better at infecting humans and other mammals. Escherichia coli, for example, is a common gut bacterium, and a darling of laboratory scientists. In its natural environments, whether the soil or the gut of a mammal, it is menaced by predatory amoebas, which threaten to engulf and digest it. In 2010, the French scientist Frantz Depaulis and colleagues found an E.coli strain called 536 that resists these predators, with genes that protect it from the amoebas’ digestive enzymes and allow it to scavenge nutrients such as iron. Rather than being disintegrated, it actually grows inside its would-be predator and eventually kills it from within.

Many of these protective genes also allow strains of the mostly harmless E.coli to cause severe illness in humans, mice and other mammals. This makes perfect sense. Many of our immune cells, like macrophages, engulf and digest microbes just as amoebas do, so an amoeba-proof bacterium is also a macrophage-proof one. By adapting to their natural predators, strains of E.coli are coincidentally pre-adapted to foil our immune system.

The coincidental evolution hypothesis might be irksome to some. What are the odds that an adaptation to one challenge would perfectly predispose an organism to another? The answer, it seems, is: pretty high. Evolution, however, is all about small probabilities manifesting through long timescales and large numbers – and microbes have both. They have been living on the planet for billions of years, and there are countless legions of them.

Casadevall likes to say that each microbe holds a different hand of cards – adaptations that allow it to cope with its environment. Most of these combinations are meaningless to us. A bacterium might be able to resist being digested by other cells, but it might not be able to grow at 37 degrees Celsius. It might grow at the right temperature, but it might not be able to tolerate our slightly alkaline pH levels. But that doesn’t matter. There are so many microbes out there that some of them will end up with a hand that lets them muscle their way into our game. ‘If you take all the microbial species in the world and imagine that they have these traits randomly, you can find pathogenic microbes for practically anything,’ says Casadevall.

we’re not central actors in the dramas that affect our lives. We are just passers-by

This inevitability paints the rise of new infectious diseases in a new light. The past few decades have seen the rise of terrifying new fungi, such as the chytrid fungus that is massacring the world’s amphibians, or the one behind the White-nose syndrome that has killed millions of North American bats. ‘People ask where these came from,’ says Casadevall. ‘It may just be that their virulence was generated by selection forces that have nothing to do with the hosts they ended up with.’

The same rationale explains why human explorers should be cautious if we ever encounter a planet with microbial life. ‘Most people in infectious disease think that if there are microbes in Mars, what do we have to worry about?’ says Casadevall. ‘They won’t have the right proteins for causing diseases in humans. But if you had enough microbes, there might be pathogens there.’

If that’s the case, these infections might continue being virulent for a long time. ‘When I was a student, parasitologists would tell you that disease was a primitive state in the relationship between a parasite and its host,’ says Levin. ‘Everyone eventually evolves to niceness and co-operation, with symbiosis and mutualism as the endpoints.’ But if virulence is coincidental to begin with, there might not be much of an evolutionary pressure for the inadvertent pathogen to change its ways.

There is something unsatisfying, almost nihilistic, about this idea. It deprives us of answers. As Casadevall wrote in a review, it says that virulence can arise by chance, ‘in a process that has no explanation, except for that it happened’. According to this outlook, we’re not central actors in the dramas that affect our lives. We’re not even bit players. We are just passers-by, walking outside the theatre and getting hit by flying props.

The most important parts of a microbe’s world are, after all, other microbes. They’ve been dealing with each other for billions of years before we came along. When we step into the crossfire of this ancient war, we risk becoming collateral damage. Like Wells’s Martians, we too can be laid low by coincidence.