Contraceptives are effective ways to prevent pregnancy, alleviate menstrual pain and ease heavy bleeding, but the hormonal changes needed to accomplish that may have some negative psychological side effects.

In a study published in the American Journal of Psychiatry, researchers in Denmark report that women taking hormonal contraceptives — like birth control pills, the patch, the ring and hormonal IUDs — have up to triple the risk of suicide as women who never took hormonal birth control.

The absolute risk of suicide associated with hormonal contraceptives is still extremely low, say the researchers, but the data suggest it’s worth studying and understanding further.

The same researchers reported last year that hormonal contraceptives were linked to a 70% higher risk of depression, which itself is associated with suicide. Because of that connection, in the current study, the researchers looked specifically at contraceptive use and suicide. They used a national study that tracked all women ages 15 and older who were living in Denmark from 1996 to 2013. The study analyzed prescriptions and filled prescriptions for contraceptives, as well as deaths and causes of death, and compared women taking this type of birth control to women who did not have a history of contraceptive use.

Get our Health Newsletter. Sign up to receive the latest health and science news, plus answers to wellness questions and expert tips. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Among women who used hormonal contraceptives currently or recently, the risk of attempting suicide was nearly double that of women who had never used contraceptives. The risk was triple for suicide. The patch was linked to the highest risk of suicide attempts, followed by IUD, the vaginal ring and then pills.

“At first I was surprised by the high risk compared to the results from the depression study I published last year,” says Charlotte Wessel Skovlund, lead author of the paper from the University of Copenhagen in Denmark. But she says that the results make sense when considering that in the current study, she compared contraceptive users to women who had never used contraceptives before, while the previous study compared them to non-users: a category that could include women who started hormonal contraceptives but stopped because of mood changes.

MORE: Teenage Girls Given Choice of Free Contraceptives Get Far Fewer Abortions

The current study also looked at a younger population, and the risk peaked in the first two months after starting a hormonal contraceptive. After a year, the risk plateaued. But even after a year, the risk remained higher than it was for women who never used contraceptives: at least double after a year and 30% higher after seven years.

The results held even after the researchers adjusted for other factors that can affect suicide risk, including mental illness and the initiation of sexual relationships.

But other scientists say the study may not have accounted for all of the potential reasons why women who use contraceptives differ from those who do not. For example, women using contraceptives are more likely to be in relationships, and that may predispose them to a higher likelihood of emotional challenges — especially for younger women, “For them, they are still more insecure in relationships and may suffer more from breakups, unhappy events and things like that,” speculates Karin Michels, professor and chair of epidemiology at University of California Los Angeles. Michels, who was not involved in the study, conducted a previous study that found a link between oral contraceptive use and heightened suicide risk among the large Nurses’ Health Study.

MORE: No, Birth Control Doesn’t Make You Have Riskier Sex

The researchers agree that the findings aren’t robust enough to discourage women from using hormonal contraceptives. But they say the results should prompt doctors to discuss the potential side effects of contraceptives and to pay more attention to women who might be at higher risk, like those who have a history of depression or mood disorders. “We think the findings are a little concerning, and we think that the consequence of these findings is that prescribers of hormonal contraceptives should make a little more effort to assess women before they get a prescription,” says Ojvind Lidegaard, the study’s senior author from the University of Copenhagen.

While it’s not clear how contraceptives may be affecting suicide risk, it’s possible that some of the risk is occurring through the way that hormones can affect mood and depression. “I believe that hormonal contraception can also have a direct effect on the brain,” says Skovlund.

The authors, as well as Michels, agree that women who are currently using contraceptives successfully should not stop using them. Rather, they suggest that women and their physicians should discuss potential suicide risk when considering contraception. “Women and their doctors should be aware about mood reactions as a potential side effect, so they can quit their hormonal contraception if they feel affected,” says Skovlund. “Doctors should be more reluctant to prescribe hormonal contraception to young girls unless there are medical reasons to do so. Other non-hormonal contraceptive options, like condoms and copper intrauterine devices [IUDs], should be considered when only contraception is needed.”

Contact us at letters@time.com.