Forty years ago President Richard Nixon declared a "war on cancer". Yet in spite of $100bn (£60bn) of taxpayer-funded research in the US alone, the cancer mortality rate remains little changed. Dozens of much-hyped "cures" developed by drug companies are either useless or have marginal effect. What can be done?

Two years ago, in a spectacularly enlightened move, the US National Cancer Institute (NCI) decided to enlist the help of physical scientists. The idea was to bring fresh insights from disciplines like physics to help tackle cancer in radical new ways. Twelve research centres were created to focus the effort, and I was approached to run the one based at Arizona State University.

With no prior knowledge of cancer, I started asking some very basic questions. What struck me from the outset is that something as pervasive and stubborn as cancer must be a deep part of the story of life itself. Sure enough, cancer is found in almost all multicellular organisms, suggesting its origins stretch back hundreds of millions of years.

Oncologists tend to think of cancer as a motley collection of cells gone berserk, but to me the way that tumours grow and spread to other organs indicates an organised and systematic strategy, designed to evade all that the body and the medical profession can throw at it. Such well-honed behaviour suggests they are the product of a long period of biological evolution.

I began wondering whether cancer might be an evolutionary throwback to the dawn of multicellular life, when single cells began cooperating and forming rudimentary aggregations. Geologists trace this phase back to a time a billion or more years ago, during the so-called Proterozoic era, long before the appearance of plants and animals with their well-ordered body plans and fully differentiated cell types. The closest living analogue might be a sponge.

How, then, might these ancestral forms reappear today inside the bodies of animals? Evolution works by building on what came before. The genes needed to fashion the primitive cellular aggregates of the Proterozoic era did not all become defunct. Some were incorporated into the genomes of later, more sophisticated, organisms, and lurk inside human beings to this day. That's because they still serve a crucial function. When an embryo develops, its genes lay down a body plan, starting with the most basic and most ancient features.

A century ago the German biologist Ernst Haekel pointed out that the stages of embryo development recapitulate the evolutionary history of the animal. Human embryos, for instance, develop, then lose, gills, webbed feet and rudimentary tails, reflecting their ancient aquatic life styles. The genes responsible for these features normally get silenced at a later stage of development, but sometimes the genetic control system malfunctions and babies get born with tails and other ancestral traits. Such anomalous features are called atavisms.

Charles Lineweaver of the Australian National University is, like me, a cosmologist and astrobiologist with a fascination for how cancer fits into the story of life on Earth. Together we developed the theory that cancer tumours are a type of atavism that appears in the adult form when something disrupts the silencing of ancestral genes. The reason that cancer deploys so many formidable survival traits in succession, is, we think, because the ancient genetic toolkit active in the earliest stages of embryogenesis gets switched back on, re-activating the Proterozoic developmental plan for building cell colonies. If you travelled in a time machine back one billion years, you would see many clumps of cells resembling modern cancer tumours.

The implications of our theory, if correct, are profound. Rather than cancers being rogue cells degenerating randomly into genetic chaos, they are better regarded as organised footsoldiers marching to the beat of an ancient drum, recapitulating a billion-year-old lifestyle. As cancer progresses in the body, so more and more of the ancestral core within the genetic toolkit is activated, replaying evolution's story in reverse sequence. And each step confers a more malignant trait, making the oncologist's job harder.

There is some good news buried in this conclusion. The ancient toolkit will be a limited set of specific genes and therefore present a well-defined target for therapy. To build up a full picture of cancers as atavisms, we have to map not just the human genome but the genomes of our oldest common multi-celled ancestors, including those of plants, insects and fungi, and work out how the cancer story relates to these life forms too. It will be in the convergence of evolutionary biology, developmental biology and cancer biology that the answer to cancer will lie. Nor will this confluence be a one-way street. By studying cancer, biologists can gain clues about how complex life evolved on Earth, and maybe on other planets too.

Cancer touches every family in one way or another. As other diseases are brought under control, cancer is set to become the number one killer, and is already in epidemic proportions worldwide. Although the elusive "cure" may be a distant dream, understanding the true nature of cancer will enable it to be better controlled and less menacing.