Wing commander Brian Liversidge was 60 when he was diagnosed, in 2004, as having aggressive prostate cancer. The CEO of a Cumbria educational trust was given 18 months to live.

Liversidge – who is married with two sons and three grandsons – has since confounded that grim prediction thanks to a sequence of remarkable medical developments that have kept him alive and which raise hopes it may soon become possible to treat many cancers as manageable chronic conditions.

However, the story also reveals the considerable pitfalls that lie ahead for scientists as they strive to achieve this goal.

Five different forms of therapy have been used – so far – to keep Liversidge alive. Now living in Colchester in Essex with his wife Susan, he recalls the battle he has fought to stave off a cancer that has since spread to his bones.

"There are side-effects with every drug you take and sometimes it is hard to separate these from the cancer's symptoms. Sometimes I have felt so bad that I have asked to be taken off my treatment. But as a father of sons, I am keen to progress treatment for prostate cancer as far as possible. I would love to hear the doctors say they have cracked it," said Liversidge who was keen to record his appreciation of the oncology teams – in Preston, Colchester and at the Marsden – who have looked after him.

The key development that has transformed scientists' strategy for battling cancer has been the recent discovery that tumours are far more genetically complex than previously realised. "By analysing cancer tissue samples from patients we have found there is an enormous genetic difference between cells found within a single tumour," said Professor Martin Gore of London's Royal Marsden hospital. "It was a surprise."

Chris Jones of The Institute of Cancer Research (ICR), London, agreed. "Until recently, it was assumed cancer cells were more or less identical clones of each other. We have found this is not true. Cells, taken from a single tumour from one person, can have many different genetic alterations within them. This presents us with a huge challenge in trying to develop treatments, though in the long term our new awareness should also provide us with an opportunity to create powerful anti-cancer drug regimes."

For most of the past 40 years, cancers have been treated by surgery, radiotherapy or chemotherapy. This last technique involves the use of cytotoxic drugs which can kill cells that they encounter. By carefully adjusting doses of these drugs, doctors have been able to kill off cancer cells while leaving normal cells unaffected – in many cases. But the considerable toxicity of chemotherapy drugs means they can only be administered for a few weeks, which limits their tumour-killing potential.

"Until the 21st century these were more or less the only drugs we had to treat cancer," said Professor Paul Workman, the ICR's deputy chief executive. "Chemotherapy gave us a one-size-fits-all way of dealing with tumours. We needed something more specific, drugs to target cancer cells but not healthy tissue. We got that chance with the completion of the human genome project."

By unravelling the structure of all the 23,000 genes in the human body, scientists have since pinpointed around 150 genes which specific mutations turn into cancer-causing agents, or oncogenes as they are also known. Some oncogenes cause cells to divide and spread uncontrollably. Others switch off the mechanisms that normally halt cell division. Others block DNA repair mechanisms that keep cells healthy. The end result is the creation of a tumour.

The discovery of these cancer-causing genes paved the way for the creation of the first targeted therapies which could pinpoint oncogenes and block their actions, thus targeting cancer cells but not cells from healthy tissue. An example is provided by vemurafenib. Given to patients with metastatic melanoma – an aggressive skin cancer that spreads quickly through the body – the drug has a remarkable effect.

"Even though the cancer has spread through a patient's body, the tumours melt away," said Workman. "Patients feel great. It looked the perfect cancer drug. Unfortunately, we have recently found this impact is only temporary. After six months or so, the tumours come back and when they do they are completely resistant to vemurafenib."

Nor is this story an isolated one. "We have seen resistance appear with almost every one of the new generation of targeted cancer drugs we have created," said Workman. "The drugs work well at first. Then the cancer returns in a form that no longer responds to the drug."

At face value, the appearance of this cancer drug resistance is extremely worrying. But doctors are quick to point out their new drugs still extend lives, although sometimes for only a brief period. "Six months for life extension is an average," said Workman. "In some cases, they survive for several years. In any case, even a brief extension to lives is still welcomed by patients."

As to the cause of the drug resistance, this is explained by the tumours' genetic complexity. This indicates that if a genetic pathway in a tumour is blocked by a new targeted drug treatment, it is very likely that another pathway will be activated. These cells will then take over and divide, re-establishing the cancer. "Tumours evolve on a very simple principle," said Workman. "It is all about the survival of the nastiest."

The idea is alarming though researchers believe their new awareness of the genetic diversity of cancers actually offers them an opportunity to tackle tumours by devising a new strategy that mimics the approach used many years ago in successfully combating HIV: simultaneously giving several different drugs, each targeting a different genetic pathway in the cancer.

"HIV has considerable genetic variety and that was causing problems for doctors," said Workman. "They got around that problem by giving patients three drugs simultaneously, a treble whammy that has worked very effectively by tackling different viral genes at the same time. We are now doing that for cancer – by giving several drugs, each one targeting a different genetic pathway inside a tumour cell."

Johann de Bono, professor of experimental cancer medicine at the ICR, said: "What we envisage is giving several drug regimes at the same time or in very tight sequences. Drug A and B one week, C and D the next week. The aim would be to use the technique to put the tumour right against the ropes and keep it there."

Gore also backed the idea, although he counselled caution. "Giving drugs simultaneously does not appear to work in every case and we may have to stagger the way we administer them. For example, with kidney cancer, we have tried to give targeted therapies at the same time but these have not worked. So now we give them in sequence. On the other hand, with melanomas we have found that combining two targeted agents at the same time does have an improved impact. So we have slightly different approaches depending on the cancer. This is how the strategy will play out in future."

INSIDE A TUMOUR

Tumour cells from an adult glioblastoma - also known as glioblastoma multiforme, or GBM. Photograph: The Institute Of Cancer Research

Cells from a single tumour reveal the different genetic pathways which glow pink and green on this photograph. The image reveals the unexpected complexity of cancer cells.

Far from being clones of each other, these cells have considerable biological variety. This diversity complicates the way in which new targeted therapies can be administered to cancer patients and suggests that several versions of these personalised medicines will have to be given at the same time, as combinations of HIV drugs are given simultaneously.

"We have pinpointed about 150 cancer-causing genes so far, but the drugs that we have on the market mainly tackle about only 15 of them," added Paul Workman of the ICR. "Many others are in the pipeline, however, and we need to get them into clinics as soon as possible."