Poppy was travelling the world when it began. It started in Malaysia and got worse in Australia. The student had the infection for a few weeks before she saw a doctor and was given a quick hit of broad-spectrum antibiotics. They didn’t work.

“I went on getting worse,” she says. “I was not in any city long enough to do a full culture and wait for the right antibiotics.”

Finally, after a month, she was given treatment that helped, but it was a temporary respite. Fifteen years later, Poppy (not her real name) is still often in pain. The infection was an attack of cystitis, a common acute urinary tract infection (UTI). Back in the UK, her condition became chronic. “I got stuck in a cycle of infection.”

Most people – and most doctors – think of UTIs as nasty but short-lived. It is unclear how they are contracted, but according to the NHS “most cases are thought to occur when bacteria that live harmlessly in the bowel or on the skin get into the bladder through the urethra (the tube that carries urine out of your body)”. The standard test is for bacteria in a urine sample. The standard treatment is three to five days of antibiotics. A one-off acute attack can be cleared very quickly. But there are thousands of women whose experience is very different. They cannot get well. They have chronic infection that dominates their lives. They spend months and years in and out of GP practices and hospitals, having tests and treatments that sometimes exacerbate the problem. And to make it so much worse, they feel they are not taken seriously.

Prof James Malone-Lee.

“One male consultant said maybe I was washing too zealously. Male GPs asked why I was crying. I’m crying because I’m in pain and I’m scared and no one seems to know what’s wrong,” says Poppy.

“I don’t understand how we can do heart transplants and land on the moon and not kill the bacteria in my bladder.”

“It’s a scandal,” agrees Jade Henderson, 27, who lives in Durham and experienced a number of acute UTIs before hers became chronic in February 2017. “I was hardly leaving the house because I was constantly feeling I needed the toilet. Some women have really bad pain. Mine was more of a constant discomfort where I couldn’t put my mind to anything apart from feeling I needed to go to the toilet.”

She was unable to work, suffering stress and anxiety. “I was in a long-term relationship at the time. A guy I’d been with for five years. It had an impact on my sex life and that relationship broke down. There were other factors. But that probably played a part.

“The mental effect was awful because I was constantly going to the doctor’s and my GP was none the wiser. I was getting urine samples sent away that were coming back normal. I saw urologists in the NHS who rolled their eyes and told me there was nothing wrong.”

Like many others, she searched for help online because GPs and urologists diagnose conditions that seem to be labels for an unfixable problem – painful bladder syndrome or interstitial cystitis. There is no offer of a cure. Many of these women – and it is usually women although not solely – converse in Facebook support groups and talk of a clinic at the Whittington hospital in north London and a consultant – Prof James Malone-Lee, who has pioneered a different approach.

Malone-Lee recently retired from the NHS, but his research, his private practice and the handful of doctors who have adopted his thinking, continue the work. The gold-standard UTI test, used since the 1950s, is no good for a chronic UTI, he says, because it does not detect bacteria hidden in the bladder’s lining. In a recent study, Dr Jennifer Rohn, principal investigator of the chronic UTI group at the Royal Free hospital and University College London, found that the test “failed spectacularly” when compared with DNA analysis, which picked up infections the standard test missed every time.

Malone-Lee put it even more strongly. “For decades, patients suffering appalling symptoms have been betrayed by a urine test that isn’t fit for purpose,” he said in December.

Some in the medical establishment are anxious about his treatment, because he gives women months or even years of antibiotics. But a published review of outcomes for 624 women treated over 10 years at the Luts (lower urinary tract symptoms) clinic at Hornsey central health centre showed that they got better, just one had a serious side-effect and that there were no instances of antimicrobial resistance. That one case of a serious side-effect resulted in the clinic being closed down for some months. A research paper subsequently found the other patients got worse during the closure.

Henderson is typical of the women treated at the clinic. “Until I found the professor, I just felt alone and wondered would I ever get better? Would I ever get back to a normal life?” she says.

What does she mean when she calls it a scandal? “The tests that the NHS use as gold standard miss 50% of infections,” she says. “The fact that this condition’s not recognised by the NHS, not really. It is so very, very common.” People with chronic UTIs become expert patients. More expert, they believe, than their doctors.

She knows of many fellow sufferers through her Facebook group, but happens across them in her everyday life, too, which makes her think it is far more usual than people realise. “I was dating a guy and told him about the problems I’d had with my bladder and he said he’d got a female friend who had been off work for about nine months with her bladder. She was really poorly, on morphine. Anyway, he put me in touch with her and she ended up going to see the professor. This was about seven months ago. She’s getting a lot better. She had ulcers in her bladder – but the antibiotics seem to be working a treat.

“There are clearly so many women out there and even specialist urologists have no knowledge of it at all. It’s trying to change beliefs that have been instilled into doctors for so many years.”

She likens Malone-Lee to “a one-man band trying to educate all these urologists and GPs. It’s going to take time. Many women are being fobbed off.”

UTIs are commonly thought to be an affliction of mostly elderly people. But there are many young women who suffer chronic infection – and even children.

Alice was just three years old when she developed a UTI, possibly from a swimming pool, says her mother, Alison Taylor, who lives in Oxford. “We went to the GP. She was given a course of antibiotics and it did clear the infection, but, 36 hours later, the symptoms were back with a vengeance.”

Alice had pain in her lower abdomen. “She wasn’t able to sleep. She was running every five minutes to the loo,” says her mother. It was distressing. “When they are tiny, you feel very responsible for them.”

Jade Henderson.

They kept going to the GP. Urine tests came back negative. “It was heartbreaking. I was desperately hoping the cultures would show something and she could be prescribed antibiotics and it would all go away,” says Taylor. But they showed “mixed growth” or “insufficient growth”, which were classified as negative results.

One time, when Alice was distressed, her mother called the NHS’s 111 medical helpline. “The call handler told me it was just the symptoms of a urinary tract infection and the GP would prescribe a course of antibiotics and everything would be all right. I just felt sick because I knew that was not the case.”

They were referred to hospital, where scans and more tests all came back negative. “We spoke to one consultant who said: ‘We think she has painful bladder syndrome. Even presented with all the classic symptoms of an infection, we were being told she doesn’t have an infection. It was so worrying and so frustrating,” Taylor says.

“The paediatric urologists were “lovely” and “wonderful”, but fixed in their assumptions. “They put so much false faith in the tests. It was like talking to a brick wall.

“It was just a terrible time,” continues Taylor. “I felt so vulnerable and incredibly worried.”

She found references online to D-mannose powder, which small studies have suggested may help prevent infection. “It was just a simple sugar. I thought it was worth trying. We didn’t have anything to lose,” she says.

Meanwhile, she had found a GP who specialised in chronic UTIs, who sent Alice’s urine to a private lab where it was cultured longer and was found to have an E-coli infection. Taylor was reluctant to have her daughter on a long course of antibiotics and the D-mannose seemed to be working, so they decided to wait. Alice was fine for three years. When the symptoms returned, and various short-course antibiotics didn’t work, she was referred to Malone-Lee.

Alice, now six, was missing school, says Taylor. “It impacts on every area of a child’s life. She was incredibly anxious and depressed. When her symptoms were really bad she said to me: ‘Mummy, I don’t want to live like this.’”

Malone-Lee saw her within days and started her on long-term antibiotics, warning that it would take time, but after nine months of treatment, she has made a full recovery. “Her quality of life is back. She is a totally different individual,” says Taylor, who now helps support other parents. Through this involvment, she says she has heard of children being prescribed morphine to control the pain. “What is happening is appalling,” she says.

Malone-Lee says that by the time patients arrive at his clinic they have, on average, been suffering from a chronic UTI for six-and-a-half years. “My experience in seeing new patients is always the same. There’s a kind of panic inside me to get this right because it’s had such a terrible toll on them,” he says. “They’re in pain all the time, they can’t have sex, their personal relationships are an awful mess, they’re so unwell many of them can’t work, go on holiday, socialise ... They know there’s something wrong, but keep being told there’s not. It’s frankly a nightmare.”

He says the flaws in current methods for diagnosis date back to some of the earliest research on UTIs in the 1950s. It was thought, incorrectly, that the normal human bladder is completely sterile. So if bacteria grew in a cultured urine sample it was thought to signal an underlying infection. In an early paper, an arbitrary threshold was put forward as being indicative of a true infection rather than contamination. And despite this threshold never having been properly validated, it gradually became accepted.

For any given patient, it is difficult to pinpoint which particular microbe is causing the problem – it may even be interactions between several. Nevertheless, Malone-Lee and his team have been investigating possible candidates and what they discovered could help explain why UTIs can be so hard to treat.

When they grew one bacterial species, called Enterococcus, in culture along with bladder cells, it displayed a range of alarming tactics to evade treatment. The bugs were sometimes seen to burrow into cells and form “nests” on the interior. They would then go into hibernation, sometimes for months at a time, during which time they would be invulnerable to attack by antibiotics. At other times, bacterial colonies morphed into biofilms that glued themselves to the outside of cells and, again, entered a temporarily dormant state.

The findings explained why short courses of treatment can be ineffective: while bugs that are floating free in the bladder are killed off, a reservoir might remain inside cells or on the surface of the bladder wall. As cells in the bladder wall are shed during the course of a natural life cycle, the dormant bacteria is released, re-activating the infection.

The bladder wall completely regenerates over a period of about nine months. And some patients, Malone-Lee argues, will only be rid of their infection by treating them for at least this long.

Instead of relying on culture tests, Malone-Lee’s lab developed a treatment regime informed by the patients’ symptoms and also the presence of white blood cells in the urine, which are normally indicative of an infection.

Typically patients are treated with antibiotics until the symptoms are gone and the white-blood-cell count returns to normal. They are then monitored to see if signs of an infection reappear. If they do, they are put back on the antibiotics. The average time for patients at the clinic to completely come off drugs is about a year.

Some very senior clinicians support Malone-Lee’s work. Mark Slack, head of gynaecology at Addenbrooke’s hospital in Cambridge, regularly sees patients debilitated by chronic UTIs, describing the condition as a “major problem”. “In 20 years we haven’t advanced one inch from where we were,” he says.

He is critical of those who dismiss patients’ problems as being primarily psychological – instead saying having a UTI for long enough can impact a person’s psychological well being. He says he has had patients who have been completely well, then get what seems to be a UTI and become virtually disabled.

Nikesh Thiruchelvam, a consultant urologist also at Addenbrooke’s, has sent some of his patients to Malone-Lee.“There are much more expensive but higher quality techniques for urine cultures that are not used in clinical practice,” he says, describing the usual tests as “outdated”.

Malone-Lee’s published data “shows quite good results” says Thiruchelvam. But it is retrospective and not a randomised trial. There is also the difficult issue of long-term antibiotics at a time when the NHS is trying to use these vital drugs less to preserve their power against bacterial resistance.

Yet there is a way forward: a clinical trial. He thinks the National Institute for Health Research, part of the NHS, should commission one – and the patient groups should push for it to happen.

Cutic, the chronic urinary tract infection campaign, run by patients and families, is calling for better testing and treatment for the half a million people in Britain it estimates are suffering from chronic UTIs. They say the 10-year study on the efficacy of long-term antibiotics for the treatment of chronic Luts was “very persuasive”.

Alison Taylor is one of those who believes the evidence is already clear.

“The international research is supportive of Malone-Lee’s findings,” she says. “He is not a lone voice at all.”