Masculinity is a delicate flower. Easily wilted, it needs constant care and nurturing, as any wife or girlfriend of a man well knows. And that may mean keeping a distance from a man’s friends.

A new study by researchers at Cornell University and the University of Chicago finds that when a guy’s female partner gets too chummy with his male buddies, he is more likely to suffer from symptoms of sexual dysfunction.

The phenomenon is called “partner betweenness,” said study authors Benjamin Cornwell and Edward Laumann, which occurs when a woman becomes closer with a man’s confidants than he is himself. About 25% of men in the study said they experienced partner betweenness in at least one of their close confidant relationships. These men were 92% more likely than men who were closer to all their friends than their partners were to suffer from erectile dysfunction (ED).

The problem is more complicated than simple jealousy. It’s not that men are worried their partners will run off with one of their buddies. Rather, it’s that men rely on their close male friendships to fuel their sense of autonomy, privacy and independence, which are in turn crucial contributors to their feelings of masculinity. When a woman steps in and usurps one of those close male bonds, she may threaten her partner’s manhood and his sense of male identity, thereby increasing the likelihood of sexual dysfunction.

MORE: One-Third of Americans Would Choose Cellphones Over Sex

The problem was most significant in the youngest men Cornwell and Laumann studied. The researchers looked at data from the 2005 University of Chicago National Social Life, Health, and Aging Project, which surveyed 3,005 men aged 57 to 85 years of age.

The association between partner betweenness and erectile dysfunction was strongest in men aged 57 to 64, the age at which many men go through big changes in life, Laumann said. In fact, in these men, partner betweenness had an association with ED that was comparable to prostate issues.

“This is when people are retiring, leaving the work force, reorganizing their life,” Laumann told the Chicago Sun-Times. “This is very threatening to a male’s understanding of himself.”

Men in the age group Cornwell and Laumann studied were already more prone to erectile dysfunction than younger men, with a third of the study participants suffering from the condition. But even after researchers accounted for health issues, like diabetes and obesity, and medications that can contribute to sexual problems, the association with partner betweenness persisted.

Even the healthiest men in the study who were capable of having robust sex lives suffered more problems in the bedroom when their partners got too close to their male friends. But the good news is, the issue appears to dissipate with age.

MORE: Scientists Restore Fertility in Mice Using Lab-Generated Sperm

Among men in the 70s and 80s, researchers found no link between erectile dysfunction and a couple’s shared friendships. That may be because men’s definition of masculinity and their self-identity evolve with age: they may go from being the hard-charging head of the family to a wise, family-oriented grandpa figure.

“Older men’s greater focus on close, kin-oriented relationships increases their likelihood of adopting new definitions of masculinity that emphasize conveying experience and mentoring rather than independence and autonomy,” Cornwell said. “Under these circumstances, partner betweenness is less likely to trigger erectile dysfunction.”

The authors emphasize that having shared companions is good for couples, contributing to a sense of “couplehood” and providing support to the relationship. In most cases, partner betweenness isn’t a problem (indeed, many women might say they have a hard enough time simply tolerating their partners’ buddies), but just in case, it might be a good idea to keep a safe distance.

The new research was published in the American Journal of Sociology.

MORE: Survey: Women Enjoy Gaming More than Sex, Play as Much as Men