Researchers say that indulging in as little as one drink a day for women and two drinks daily for men can boost the risk of breast, colon, oral, liver and esophageal cancers. But the risk was higher for men who smoked, even those who had quit, than for non smokers.

Scientists at the Harvard T. H. Chan School of Public Health and Brigham and Women’s Hospital report in the BMJ on a review of nearly 136,000 men and women enrolled in studies that followed their health-related behaviors and outcomes for 30 years. Overall, those who drank more showed a higher risk of alcohol-related cancers, which wasn’t surprising, given that over-indulging can promote abnormal growths in certain organs like the liver.

MORE: 5 Things You Need to Know About Alcohol, Backed By Research

But what was more surprising was that, according to their observations, it didn’t take much. Previous studies have focused on heavy drinking, while the current analysis looked at light to moderate drinking. Among women, up to one drink a day contributed to a 13% higher risk of developing alcohol-related cancers, primarily breast cancer. For men, up to two drinks a day also increased the risk of certain cancers, but only for those who had smoked. Non smoking men didn’t show any higher risk.

The results suggest that smoking may be an important contributor to certain cancers, especially in combination with alcohol. In fact, says Yin Cao, a post doctoral research fellow at the School of Public Health and lead author of the study, it’s enough to consider becoming even stricter when it comes to imbibing. “For men, especially those who ever smoked, they should limit alcohol to even below the recommended limit,” she says. “And smoking and heavy alcohol consumption should be absolutely avoided to prevent cancer.”

Get our Health Newsletter. Sign up to receive the latest health and science news, plus answers to wellness questions and expert tips. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Contact us at letters@time.com.