To be a homophobe in 2014 is, increasingly, to find oneself on the fast track to social scorn. In an environment of growing acceptance, we condemn homophobic feelings, particularly in men, because we think they come from inside the individual and are thus his full responsibility. A man who says hateful things about gays is “backward.” He’s protecting his social status, or maybe he’s secretly gay himself. He needs to grow up or come out already.

However, the continued existence of homophobia—despite the obvious downsides—raises questions about its basic nature: Do psychological theories like those above really explain why gayness, specifically, evokes such fear, the kind that can sometimes even lead to violent speech and action? Do they account for why homophobia is such an easy bulwark against masculine insecurity? Why does coming out seem so impossible to some men? The only way to answer these questions is to stop thinking of homophobia as a personal choice and understand it as the inevitable and deliberate result of the culture in which American men are raised.

Clearly, men in America have grown up learning to be scared of gayness. But not only for the reasons we typically think—not only, in the end, because of religion, insecurity about their own sexuality, or a visceral aversion to other men’s penises. The truth is, they’re afraid because heterosexuality is so fragile.

Heterosexuality’s power lies in perception, not physical truth—as long as people think you’re exclusively attracted to the right gender, you’re golden. But perception is a precarious thing; a “zero-tolerance” policy has taught men that the way people think of them can change permanently with one slip, one little kiss or too-intimate friendship. And once lost, it can be nearly impossible to reclaim.

Put another way, the zero-tolerance rule means that if a man makes one “wrong” move—kisses another man in a moment of drunken fun, say—he is immediately assumed to be gay. Women have a certain amount of freedom to play with their sexuality (mostly because society has a hard time believing in lesbian sex at all). Male sexuality, on the other hand, is understood as unidirectional. Once young men realize they are gay, they become A Gay Person. We don’t hear about gay men discovering an interest in women later in life, and we rarely believe men when they say they are bisexual—the common, if erroneous, wisdom is that any man who says he is bi is really just gay and hasn’t admitted it yet.

The result of all this is that men are not allowed “complex” sexualities; once the presumption of straightness has been shattered, a dude is automatically gay. That narrative does not allow much freedom to explore even fleeting same-sex attractions without a permanent commitment. I knew a guy who, straight in high school, hooked up with dudes for the first semester of college. He was then in a monogamous relationship with a woman for the rest of college; in the weeks before graduation, I would still hear people express confusion about the existence of their relationship.

The zero-tolerance policy is legitimately scary, then, not just because it sticks you with a label, but also because it erases a lifetime of straightness. One semester of experimentation was worth more than every other hook-up and romance of this guy’s life—both before and afterward.

Indeed, such erasure is scary even if homosexuality itself isn’t a bad thing. Even if religion and Esquire didn’t teach men to be scared of each other’s bodies, they would still be afraid of the way a brush with gayness can so suddenly erase the rest of their sexuality. With so much on the line, it’s no surprise that men take up the job of policing this boundary themselves, lest it be policed by someone else, to their detriment.

It’s worth noting that men confront their fear with brilliant creativity. High-schoolers accuse each other, their activities, and even objects of being gay with precisely the zero-tolerance attitude that they themselves are navigating. A popular game in high school was “fag tag,” where boys slap each other’s packages with the back of their hands. In college they played chicken, where two guys each slide their hand up the other one’s inner thigh. Whoever gets freaked out first loses—or wins, really. These games aren’t just grounded in disgust with homosex; they are playing out exactly what society has taught men about heterosexuality: One wrong move, and you’ll be permanently marked.

Homophobia, then, is precisely a fear, and one that these men are not at all foolish for entertaining. The behavior it engenders is a perceptive response to a sick system, rather than a sickness itself. That’s why I don’t hold a grudge against the kids in high school who said “fag,” or the occasional bartender who makes a weird comment about my date—they’re understandably more scared of me than I am of them.