Every year, around about this time, thousands of European robins escape the oncoming harsh Scandinavian winter and head south to the warmer Mediterranean coasts. How they find their way unerringly on this 2,000-mile journey is one of the true wonders of the natural world. For unlike many other species of migratory birds, marine animals and even insects, they do not rely on landmarks, ocean currents, the position of the sun or a built-in star map. Instead, they are among a select group of animals that use a remarkable navigation sense – remarkable for two reasons. The first is that they are able to detect tiny variations in the direction of the Earth’s magnetic field – astonishing in itself, given that this magnetic field is 100 times weaker than even that of a measly fridge magnet. The second is that robins seem to be able to “see” the Earth’s magnetic field via a process that even Albert Einstein referred to as “spooky”. The birds’ in-built compass appears to make use of one of the strangest features of quantum mechanics.

Over the past few years, the European robin, and its quantum “sixth sense”, has emerged as the pin-up for a new field of research, one that brings together the wonderfully complex and messy living world and the counterintuitive, ethereal but strangely orderly world of atoms and elementary particles in a collision of disciplines that is as astonishing and unexpected as it is exciting. Welcome to the new science of quantum biology.

Most people have probably heard of quantum mechanics, even if they don’t really know what it is about. Certainly, the idea that it is a baffling and difficult scientific theory understood by just a tiny minority of smart physicists and chemists has become part of popular culture. Quantum mechanics describes a reality on the tiniest scales that is, famously, very weird indeed; a world in which particles can exist in two or more places at once, spread themselves out like ghostly waves, tunnel through impenetrable barriers and even possess instantaneous connections that stretch across vast distances.

But despite this bizarre description of the basic building blocks of the universe, quantum mechanics has been part of all our lives for a century. Its mathematical formulation was completed in the mid-1920s and has given us a remarkably complete account of the world of atoms and their even smaller constituents, the fundamental particles that make up our physical reality. For example, the ability of quantum mechanics to describe the way that electrons arrange themselves within atoms underpins the whole of chemistry, material science and electronics; and is at the very heart of most of the technological advances of the past half-century. Without the success of the equations of quantum mechanics in describing how electrons move through materials such as semiconductors we would not have developed the silicon transistor and, later, the microchip and the modern computer.

However, if quantum mechanics can so beautifully and accurately describe the behaviour of atoms with all their accompanying weirdness, then why aren’t all the objects we see around us, including us – which are after all only made up of these atoms – also able to be in two place at once, pass through impenetrable barriers or communicate instantaneously across space? One obvious difference is that the quantum rules apply to single particles or systems consisting of just a handful of atoms, whereas much larger objects consist of trillions of atoms bound together in mindboggling variety and complexity. Somehow, in ways we are only now beginning to understand, most of the quantum weirdness washes away ever more quickly the bigger the system is, until we end up with the everyday objects that obey the familiar rules of what physicists call the “classical world”. In fact, when we want to detect the delicate quantum effects in everyday-size objects we have to go to extraordinary lengths to do so – freezing them to within a whisker of absolute zero and performing experiments in near-perfect vacuums.

Quantum effects were certainly not expected to play any role inside the warm, wet and messy world of living cells, so most biologists have thus far ignored quantum mechanics completely, preferring their traditional ball-and-stick models of the molecular structures of life. Meanwhile, physicists have been reluctant to venture into the messy and complex world of the living cell; why should they when they can test their theories far more cleanly in the controlled environment of the lab where they at least feel they have a chance of understanding what is going on?

Erwin Schrödinger, whose book What is Life? suggested that the macroscopic order of life was based on order at its quantum level. Photograph: Bettmann/CORBIS

Yet, 70 years ago, the Austrian Nobel prize-winning physicist and quantum pioneer, Erwin Schrödinger, suggested in his famous book, What is Life?, that, deep down, some aspects of biology must be based on the rules and orderly world of quantum mechanics. His book inspired a generation of scientists, including the discoverers of the double-helix structure of DNA, Francis Crick and James Watson. Schrödinger proposed that there was something unique about life that distinguishes it from the rest of the non-living world. He suggested that, unlike inanimate matter, living organisms can somehow reach down to the quantum domain and utilise its strange properties in order to operate the extraordinary machinery within living cells.

Schrödinger’s argument was based on the paradoxical fact that the laws of classical physics, such as those of Newtonian mechanics and thermodynamics, are ultimately based on disorder. Consider a balloon. It is filled with trillions of molecules of air all moving entirely randomly, bumping into one another and the inside wall of the balloon. Each molecule is governed by orderly quantum laws, but when you add up the random motions of all the molecules and average them out, their individual quantum behaviour washes out and you are left with the gas laws that predict, for example, that the balloon will expand by a precise amount when heated. This is because heat energy makes the air molecules move a little bit faster, so that they bump into the walls of the balloon with a bit more force, pushing the walls outward a little bit further. Schrödinger called this kind of law “order from disorder” to reflect the fact that this apparent macroscopic regularity depends on random motion at the level of individual particles.

But what about life? Schrödinger pointed out that many of life’s properties, such as heredity, depend of molecules made of comparatively few particles – certainly too few to benefit from the order-from-disorder rules of thermodynamics. But life was clearly orderly. Where did this orderliness come from? Schrödinger suggested that life was based on a novel physical principle whereby its macroscopic order is a reflection of quantum-level order, rather than the molecular disorder that characterises the inanimate world. He called this new principle “order from order”. But was he right?

Up until a decade or so ago, most biologists would have said no. But as 21st-century biology probes the dynamics of ever-smaller systems – even individual atoms and molecules inside living cells – the signs of quantum mechanical behaviour in the building blocks of life are becoming increasingly apparent. Recent research indicates that some of life’s most fundamental processes do indeed depend on weirdness welling up from the quantum undercurrent of reality. Here are a few of the most exciting examples.

Enzymes are the workhorses of life. They speed up chemical reactions so that processes that would otherwise take thousands of years proceed in seconds inside living cells. Life would be impossible without them. But how they accelerate chemical reactions by such enormous factors, often more than a trillion-fold, has been an enigma. Experiments over the past few decades, however, have shown that enzymes make use of a remarkable trick called quantum tunnelling to accelerate biochemical reactions. Essentially, the enzyme encourages electrons and protons to vanish from one position in a biomolecule and instantly rematerialise in another, without passing through the gap in between – a kind of quantum teleportation.

And before you throw your hands up in incredulity, it should be stressed that quantum tunnelling is a very familiar process in the subatomic world and is responsible for such processes as radioactive decay of atoms and even the reason the sun shines (by turning hydrogen into helium through the process of nuclear fusion). Enzymes have made every single biomolecule in your cells and every cell of every living creature on the planet, so they are essential ingredients of life. And they dip into the quantum world to help keep us alive.

Another vital process in biology is of course photosynthesis. Indeed, many would argue that it is the most important biochemical reaction on the planet, responsible for turning light, air, water and a few minerals into grass, trees, grain, apples, forests and, ultimately, the rest of us who eat either the plants or the plant-eaters.

The initiating event is the capture of light energy by a chlorophyll molecule and its conversion into chemical energy that is harnessed to fix carbon dioxide and turn it into plant matter. The process whereby this light energy is transported through the cell has long been a puzzle because it can be so efficient – close to 100% and higher than any artificial energy transport process.

Sunlight shines through chestnut tree leaves. Quantum biology can explain why photosynthesis in plants is so efficient. Photograph: Getty Images/Visuals Unlimited

The first step in photosynthesis is the capture of a tiny packet of energy from sunlight that then has to hop through a forest of chlorophyll molecules to makes its way to a structure called the reaction centre where its energy is stored. The problem is understanding how the packet of energy appears to so unerringly find the quickest route through the forest. An ingenious experiment, first carried out in 2007 in Berkley, California, probed what was going on by firing short bursts of laser light at photosynthetic complexes. The research revealed that the energy packet was not hopping haphazardly about, but performing a neat quantum trick. Instead of behaving like a localised particle travelling along a single route, it behaves quantum mechanically, like a spread-out wave, and samples all possible routes at once to find the quickest way.

A third example of quantum trickery in biology – the one we introduced in our opening paragraph – is the mechanism by which birds and other animals make use of the Earth’s magnetic field for navigation. Studies of the European robin suggest that it has an internal chemical compass that utilises an astonishing quantum concept called entanglement, which Einstein dismissed as “spooky action at a distance”. This phenomenon describes how two separated particles can remain instantaneously connected via a weird quantum link. The current best guess is that this takes place inside a protein in the bird’s eye, where quantum entanglement makes a pair of electrons highly sensitive to the angle of orientation of the Earth’s magnetic field, allowing the bird to “see” which way it needs to fly.

All these quantum effects have come as a big surprise to most scientists who believed that the quantum laws only applied in the microscopic world. All delicate quantum behaviour was thought to be washed away very quickly in bigger objects, such as living cells, containing the turbulent motion of trillions of randomly moving particles. So how does life manage its quantum trickery? Recent research suggests that rather than avoiding molecular storms, life embraces them, rather like the captain of a ship who harnesses turbulent gusts and squalls to maintain his ship upright and on course.

Just as Schrödinger predicted, life seems to be balanced on the boundary between the sensible everyday world of the large and the weird and wonderful quantum world, a discovery that is opening up an exciting new field of 21st-century science.

Life on the Edge: The Coming of Age of Quantum Biology by Jim Al-Khalili and Johnjoe McFadden will be published by Bantam Press on 6 November. Click here to buy it for £15