Get breaking news alerts and special reports. The news and stories that matter, delivered weekday mornings.

Breast cancer can “smolder” and return even 20 years later unless patients keep taking drugs to suppress it, researchers reported Wednesday.

They were looking for evidence that at least some breast cancer survivors might be able to skip the pills that reduce the risk of the breast tumors coming back, but found that even women with “low-risk” cancers had a small rate of recurrence 15 and 20 years later.

This means women with the most common type of breast cancer, called estrogen-positive or hormone-positive breast cancer, need to think carefully about whether they want to stop taking the pills, even if they cause side-effects, doctors said.

“These breast cancers have a lingering smoldering quality and carry substantial risk of late recurrence after five years of therapy,” said Dr. Harold Burstein of the Dana Farber Cancer Institute, who was not involved in the study.

“Many patients think. ‘OK, I made it to five years. I know I’m safe’,” said Dr. Jennifer Litton, an oncologist at the MD Anderson Cancer Center. “But for estrogen-receptor positive breast cancer, it’s a continued lifelong risk.”

Related: Breast Cancer Drug Can Also Prevent Cancer

Breast cancer is the second-biggest cancer killer of American women, after lung cancer. The American Cancer Society says every year, it's diagnosed in 200,000 women and a few men, and kills around 40,000.

Most breast cancers are fueled by estrogen, and drugs called hormone blockers are known to cut the risk of recurrence in such cases.

Melita Keith, 45, of Victoria, Texas with her husband John, son Jackson and daughter Lucy. Keith is taking pills to keep her breast cancer from coming back and a new study suggests she may do well to keep taking them longer than she'd like to. Courtesy Melita Keith

Tamoxifen long was the top choice, but newer drugs called aromatase inhibitors — sold as Arimidex, Femara, Aromasin and in generic form — do the job with less risk of causing uterine cancer and other problems. The longer women take them, the lower their risk of having the cancer come back.

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

However, they do cause side-effects.

“My colleagues and I say that at the end of five years, the women divide themselves into two groups: those who can’t wait to get off it, and those who are terrified of getting off it and have tolerated it well,” said Dr. Eric Winer of the Dana-Farber Cancer Institute.

Related: Gel Works As Well As Pill to Prevent Breast Cancer

Melita Keith, 45, of Victoria, Texas says she’s been putting up with aches and pains after taking first tamoxifen and then Aromasin.

“I’ve been having pretty severe joint pains, hot flashes, funny trigger pinkies. When I get out of bed I feel like I’m 90. My joints ache,” said Keith, who was diagnosed with breast cancer when she was 41. “On tamoxifen, I got hot flashes, night sweats. (They were) pretty strong. On a scale of one to 10, I was 7.”

Keith, a patient of Litton’s, wanted to stop. “But if Dr. Litton tells me I need to be on it, I’m staying on it. I need to be there for my kids,” added Keith, who has a daughter in 6th grade and a son in 12th grade.

Litton and Keith plan to work together to watch symptoms over the years and decide how long Keith will continue taking the pills.

“The idea of being considered cured if you’re cancer-free for five years is not true for estrogen-positive breast cancer, although it is true for triple-negative."

Doctors have known that breast cancer can come back in a percentage of survivors. Hongchao Pan of the University of Oxford, Dr. Daniel Hayes of the University of Michigan Comprehensive Cancer Center and colleagues looked at medical records for more than 80,000 women with certain types of breast cancer who were scheduled to take tamoxifen or similar drugs for at least five years.

“Even after 5 years of adjuvant endocrine therapy, women with ER-positive, early-stage breast cancer still had a persistent risk of recurrence and death from breast cancer for at least 20 years after the original diagnosis,” they wrote in their report, published in the New England Journal of Medicine.

“Even though these women remained free of recurrence in the first five years, the risk of having their cancer recur elsewhere (for example in the bone, liver or lung) from years five to 20 remained constant,” Hayes said.

Related: Exercise Keeps Breast Cancer Survivors Alive Longer

Among women who had no breast cancer spread to nearby lymph nodes when they were first diagnosed, there was a 22 percent risk the cancer would come back after 20 years. The risk was 31 percent in women who had the cancer spread to at least three lymph nodes and 52 percent in those whose cancer had spread to up to nine nodes.

“The idea of being considered cured if you’re cancer-free for five years is not true for estrogen-positive breast cancer, although it is true for triple-negative," said Winer.

These numbers may not apply to women diagnosed with breast cancer now, the experts noted. Treatments are better now and will likely reduce the risk of cancer coming back.

“I don't want a women to hear this study and think they are destined for their cancer to come back,” said Litton.

Related: Drink Less, Exercise More to Prevent Breast Cancer

Winer said breast cancer survivors need to keep an eye on their symptoms.

“If a woman develops ill-defined symptoms at year 12, and she’s being taken care of by someone not familiar with breast cancer it may take six months to get to the bottom of it,” he said.