As film genres go, pornography is the most divisive; few art forms elicit such ardently different feelings from critics and fans. Now a new study suggests that pornography may actually physically divide people too, namely those who are married.

Married people who start watching porn are twice as likely to be divorced in the the following years as those who don’t. And women who start watching porn are three times as likely to split, according to a working paper presented at the American Sociological Association on Aug. 22. However, porn appears to have a less negative impact on marriage if couples watch it together.

The paper also finds that stopping porn-watching lowers the likelihood of divorce for women, though not for men.

While porn’s effects on relationships has been much discussed in academic literature, and even this magazine, this is the first study—if its findings hold up under peer review—that traces the effect on marital stability.

The authors used longitudinal data from the General Social Survey, which tracks, among other things, marital happiness, porn-consumption and marital status. It analyzed results from more than 2000 participants over three time periods, focusing in on participants whose porn-watching habits altered during that period. That is, the individuals did not watch pornography when first interviewed but had taken it up by the time of their second interview, or they did watch during their first interview but had given it up by the second.

Read More: How Porn is changing a generation of girls

The analysis found that 11% of people who started to watch porn between the first two time periods were divorced by the second time they were interviewed. This compares to 6% of people whose porn watching habits were unchanged, but who were like the new porn-fans in every other way. Among women who started watching porn solo, the proportion who divorced was 16%, or almost three times as much.

Conversely, female porn watchers who gave up the genre were only about as third as likely to be divorced as those who kept up the habit. Male abstainers’ chances of getting unhitched were not that different from guys who kept up the habit, although the authors caution that so few men give up porn that the sample size is too small to be reliable.

The findings also suggest that porn’s effect on marriage appears to be strongest among younger, less religious people who initially report higher levels marital happiness.

Could it be that people started watching porn because their marriages were already unhappy? “We don’t think it’s the relationship quality leading to the porn use and divorce,” says says lead author Samuel Perry, an assistant sociology professor at the University of Oklahoma, because this is data taken over time and not just a snapshot. “We are pretty confident about establishing the directional effects.”

Read More: How to stay married

Perry could not definitively explain why the impact was so much stronger on women than men, since that ran counter to previous scholarship on the issue. “That’s a bit surprising because everything else I’ve seen on porn use in relationships suggests that men’s marriages are more negatively affected by their porn use,” he says, “primarily because they’re using it more often for the purposes of masturbation rather than intimacy.”

Previous studies have found that porn has an accelerating effect on a deteriorating marriage: husbands in poor relationships tend to consume more sexually explicit material and consuming more sexually explicit material also leads to poorer relationships. Some sociologists have speculated that men turn to porn as a way of lifting their mood about their difficult home life and that the porn then becomes an easier route to sexual satisfaction than being with their partner, so they disinvest in the marriage.

It’s worth noting that Perry is also a member of the religious faculty at Oklahoma. Might his beliefs be coloring his attitude towards explicit sexual content? While he says he’s not trying to ban porn, “I certainly have moral beliefs about whether I’d want my kids to watch porn. Or my wife. But you counter that by subjecting your data to scrutiny, which I’ve done. I’ve sought to remain as neutral as possible.”

Perry’s findings also run counter to another recent paper out of the University of Western Ontario, which found that large fraction of people in a relationship who used porn reported that it had no ill-effects. In a survey of 430 people who were asked open-ended questions about their or their partner’s pornographic use, the most common response was that it had no negative impact.

Read More: How to talk to your kids about porn

“While a similar number of positive and negative perceived effects were identified, generally speaking, positive effects of pornography use were reported more frequently than negative consequences by participants,” says the study, which was published in the Archives of Sexual Behavior, “and there was a predominant tendency for participants to reject the view that pornography contributes to negative consequences.”

Some of the ways in which watching porn had a positive effect were that partners learned about their likes and dislikes, could talk more openly about sex and enhanced their intimacy. Women also said their partners’ porn use took some of the sexual burden off them. Negative effects were reported too, including the development of unrealistic expectations and feelings of jealousy. “All of these issues seem rather obvious in hindsight,” says the lead author Taylor Kohut, a postdoctoral fellow in psychology. “But here’s the thing, these perceived effects of pornography use are not really being studied in a serious way. They’re just not on the radar.”

While that study was of real people, and not just numbers, the sample size is small and was recruited by an ad that asked for people to talk about pornography and their relationship, which may mean that it drew a population more comfortable with their and their partner’s porn-watching habits in the first place, and therefore less likely to report negative effects.

Perry believes that in the context of relationship, rather than in secret and with masturbation, porn-watching may have a different effect. “My research suggests that the isolation and shame are a big part of the problem.”

Get our Health Newsletter. Sign up to receive the latest health and science news, plus answers to wellness questions and expert tips. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Contact us at letters@time.com.