One of health service's most senior figures warns there needs to be a serious rethink of how patients are cared for

The soaring number of people with long-term medical conditions such as diabetes and dementia is threatening to "overwhelm" the NHS, one of the health service's most senior figures warns.

The challenges posed by patients with chronic medical conditions are so great that they represent the "healthcare equivalent to climate change" and must force the NHS to undertake a major rethink of how it cares for such patients, Dr Martin McShane says in an interview with the Guardian.

Looking after the 15.4 million people in England with at least one long-term condition already takes up 70% of the NHS's £110bn budget – £77bn – as well as £10.9bn of the £15.5bn spent on social care in England, he says. The costs are so huge that the NHS could become unsustainable unless it gives those with long-term conditions better care, with much of it provided by GPs performing enhanced roles rather than hospital doctors, says McShane, NHS England's national director for people with long-term conditions.

McShane is responsible for those ongoing illnesses or diseases that see patients become regular users of NHS services, through check-ups, tests and operations. They include arthritis, heart disease, breathing problems, obesity and mental health conditions such as depression. Their numbers have risen dramatically in recent years, largely as a result of the ageing population and lifestyle factors such as smoking, drinking and overeating.

"I would say it's the healthcare equivalent to climate change. It is putting pressure into the system, which, unless we change the way we address the problems, will overwhelm the system," says McShane.

"This is the biggest problem facing the health system and the care system and the costs are growing year on year. They are huge already and they will continue to grow."

The NHS in its current form is not well set up to look after patients who are medically complicated, especially if they have several long-term conditions, such as arthritis, heart failure and the early signs of dementia, McShane says. While the total number of people with long-term conditions is expected to stay at around 15 million, the number with three or more conditions is expected to rise from 1.9 million to 2.9 million by 2018.

"People with multiple long-term conditions often fall through the gaps as their secondary [hospital] care is highly specialised and their GP care highly generalised, with little continuum between the two, meaning those with multiple long-term conditions can fall through the gaps when confronted with confusing and fragmented secondary care," he says.

The failure so far to reorganise services for such patients means too many are not getting proper care and can end up having largely avoidable spells in hospital, which adds to the pressure on A&E units and hospital beds and also wastes vital NHS resources.

The NHS will not bridge the £30bn gap that it fears will have emerged by 2020 between its budget and the volume of care needed unless it takes radical steps to cope with rising numbers of long-term conditions, McShane warns.

Research by Professor Andrew Street, a health economist at York University, has found that while a healthy patient costs the NHS about £288 a year, those with one long-term condition cost an estimated £783, those with two cost £1,521 and those with three cost £2,559 each. Their need for frequent treatment and monitoring means that the small minority with five such conditions cost £5,512 a year and those with six about £8,083.

England is regarded as world-leading in long-term condition management, for example by helping diabetics avoid undergoing an amputation, of a finger, toe or even a limb, as a result of complications. But despite that, says McShane: "This huge need has emerged and we haven't evolved fast enough to meet it.

"General practice is doing a fantastic job, as it always has done, and specialists have become more and more specialised. But there's the needs of a new generation, the geriboomers, who are now living longer and collecting long-term conditions, so our thinking has to change."

The NHS is still set up to deal with 20th-century medical need and must evolve rapidly to better handle long-term conditions, the greatest challenge of this century, he says.

McShane wants some family doctors to do extra training and become "complex care GPs", to look after only people with long-term conditions, especially the 5% of the population who are the heaviest users of NHS services and take up most of doctors' and nurses' time. They would then lead small teams of health and social care professionals who would try to keep the patients as well as possible in their own homes. As many as 50 GP practices could come together to do that, as well as looking after elderly patients in care homes and using telehealth to monitor patients who are still living at home, he says.

Professor Chris Ham, chief executive of the King's Fund health thinktank, said: "The NHS and social care have been slow to rise to the twin challenges of an ageing population and increased prevalence of long-term conditions like diabetes. There is now an urgent need to transform how GPs treat people with these conditions, and to support people themselves to take more control over their health."

A few innovative GPs are already undertaking specialist training to expand their skills and offering extra services in their surgeries such as minor injury clinics, diagnostic facilities and access to specialist nurses and doctors, he added.

Norman Lamb, the care and support minister, agreed with McShane that services for those with long-term conditions must improve. "We want to build a fairer society, and that means providing better care to people with long-term conditions so that they are able to enjoy an independent, fulfilling life, and have the support needed to manage their health," he said.

The government's planned £3.8bn-a-year Better Care Fund, which starts in April 2015, will fund the integration of health and social care services so patients can live independently for as long as possible, while recent changes to the GP contract should help reduce avoidable attendances at A&E units, he added.