Racked is no longer publishing. Thank you to everyone who read our work over the years. The archives will remain available here; for new stories, head over to Vox.com, where our staff is covering consumer culture for The Goods by Vox. You can also see what we’re up to by signing up here.

Not once have I had a guy who, after offering to make breakfast in the morning, stood up, stretched, and grabbed one of my shifts off the floor so he didn’t have to fry up a couple of frittatas in just his socks. Never has a man walked from my room with a dress skimming the tops of his hairy thighs, the short hem flashing cheek as he rooted around for pans, the strap falling all come-hither-like down his shoulder — and me watching all of this from my bed, biting my fist.

We’ve seen this same scenario play out a hundred times over with women wearing men’s shirts, but never really the other way around, at least in the United States. And you have to wonder: why not?

This observation isn’t anything new. We’ve been grappling with these imaginary lines for a long time now, and always end the conversation in the same stalemate. In 1938, for example, a mother wrote to her local paper asking what she should do about her son. He went to a costume party dressed as a girl for a laugh but hadn’t taken off the dresses since.

“His sisters have to keep their closets and their bureau drawers locked up to keep him from wearing their things. We have tried every way in the world to shame him and his father has thrashed him several times about it, but nothing stops him. What can we do?” she asked.

“Isn’t it queer that for a boy to want to be a girl, and look like a girl, and dress like a girl is so unusual that it fills his parents with fear that he is abnormal, whereas virtually every girl in the world wishes she were a boy?”

The response back was surprisingly introspective. The advice columnist wrote, “Isn’t it queer that for a boy to want to be a girl, and look like a girl, and dress like a girl is so unusual that it fills his parents with fear that he is abnormal, whereas virtually every girl in the world wishes she were a boy and the majority of them try to look like boys, and act like boys, and dress like boys? The greatest insult you can offer a man is to call him effeminate, but women esteem it a compliment to be told they have a boyish figure and that they have a masculine intellect.”

The reason for that has to do with the way the gender binary is enforced, and how our choice in clothing is us “doing gender.” According to Sarah Fenstermaker, the recently retired director of the University of Michigan’s Institute for Research on Women and Gender, gender is a set of behaviors, ways of being, and ways of interacting that convince ourselves and everyone around us that, deep down, we are just what we appear to be.

More than that, the binary is built on the idea that it’s 100 percent natural and, because of that, is “naturally” recognizable. To be feminine means to be the opposite of masculine, and to be masculine means to be the opposite of feminine. Period.

“When we embrace something as only ‘natural,’ it means that it can’t really be changed — that it’s baked into who we are. Anyone then who strays too far from expectations that surround this naturalness is odd, deviant, and often deserving of punishment or exclusion,” Fenstermaker explains.

To be a man and want to wear feminine flounces puts a crack in the theory that these classifications are inherent, which makes you question just how natural the power that comes with masculinity is. And in a male-dominated society, that question is a big deal. Which is why we weed out and ostracize anyone who deviates — femme gay men, butch lesbians, nonbinary individuals, trans people, and straight men who like skirts.

“The display of skirts on men is effectively an undermining of male power — by males. To put it extremely, they are like deserting troops.”

“The display of skirts on men is effectively an undermining of male power — by males. To put it extremely, they are like deserting troops. So what do we do in response? We make them gay,” Fenstermaker says. This stops the hierarchy from toppling because we reason that gay men aren’t “real” men because “real” men aren’t feminine. While it’s true that not all gay men are feminine and all lesbians are masculine, that’s the expectation used to write them off.

From an agender California teen being hospitalized for three weeks after a classmate set their skirt on fire after mistaking them for a gay man, to a high school student getting suspended for “attempting to incite a riot” for wearing a pink tutu for breast cancer awareness month (after being questioned if he was gay,) to Young Thug getting whooped by his father for wearing his sister’s glitter shoes at 12 years old, straying from your binary path has consequences, and men are constantly being reminded of that.

“Any expression of femininity results in a judgment that one is not a real man, and that is just a short step to not really being male,” Fenstermaker explains. That fear alone makes many straight men second-guess reaching for a mini.

But why were women able to put on pants seemingly scot free? Granted, it didn’t exactly happen overnight. In the beginning, there was pushback because of the power grab it hinted at — from Victorian women who went outside in bloomers getting rocks thrown at them by angry men, to Vogue calling women who kept their pants on after their factory shifts in the 1940s “slackers in slacks,” to a socialite being asked to walk to her restaurant table in nothing but her tuxedo jacket because pants weren’t dress-code approved, there were moments of backlash.

But women in button flies were accepted fairly easily, and the reason has to do with this power balance we’ve created, which doesn’t make pants and skirts equivalent. “They don’t have equivalent power, or potency, or symbolism,” Jo Paoletti, who has spent thirty years researching and writing about gender differences in American clothing and is the author of Pink and Blue: Telling the Boys from the Girls in America, shares. Masculinity is valued — it’s associated with seriousness, power, credibility, and authority, so a woman reaching into a man’s wardrobe is seen as aspirational, and it gives her leeway to play with the pieces.

But only to an extent. There is one important caveat to the borrowed look: A woman could emulate a man, but she couldn’t dress like one to a T. She had to soften the outfit with feminine touches, and if she didn’t, she was either ostracized (the way butch women and gender fluid people are) or infantilized.

A good example of this in action is the woman’s business suit in the ’80s. As John Molloy wrote in his 1977 style guide, The Woman’s Dress for Success Book, dressing like too much of a man was kind of like “a small boy who dresses up in his father’s clothing. He is cute, not authoritative.” He went on to explain: “My research indicates that a three-piece pinstripe suit not only does not add to a woman’s authority, it destroys it. It makes her look like an ‘imitation man.’”

Why? Because women could aspire to look like the men in the corner offices, but they couldn’t actually become them. No one was going to mistake a woman dressed in Brooks Brothers for an actual man, much like no one was going to mistake a little girl putting on her mom’s heels for an adult with a checkbook. And since the boxy suits only accentuated the “smallness” of the woman wearing them (and, in turn, the natural bigness of the man it belonged to in the first place,) it only made her seem more female.

These mental gymnastics that society goes through to keep the genders distinct from each other serves a very specific purpose: to keep that binary hierarchy in tact.

“[Pants and skirts] don’t have equivalent power, or potency, or symbolism.”

“Women have a role to play, which is to be the counterpart. Women only work as the counterpart if they are distinct to what they’re the counterpart to.” Marjorie Jolles, the women’s and gender studies director at Roosevelt University, explains. And our need to know gender reveals the power dynamic that comes with it. How do you treat this person underneath the clothes: with authority, or subordination?

Which leads us right back into why we don’t see men wearing this season’s knife-pleat skirts or sequined minis while out grocery shopping or drinking scotch at a bar. “Feminine clothing has absolutely no social capital for a man to put on because he’s gesturing towards a set of traits that our society doesn’t really value,” Jolles says. He’s gone from the top of the social ladder to the bottom, and that display of willingly cashing in your power is what makes the look so uncomfortable or shocking.

“It’s not a 1:1 comparison,” Jolles explains. “The woman is gesturing using the codes of the ruling class: men. A man gesturing to the codes of the oppressed class gets him nothing, except ridicule.”

This ridicule is obviously one of the main reasons why most straight men won’t put on shifts, but many also won’t play with femininity even in private, where there’s no one to judge. We self-police just as often as others police us.

“The woman is gesturing using the codes of the ruling class: men. A man gesturing to the codes of the oppressed class gets him nothing, except ridicule.”

Fashion isn’t just a public manifestation of gender, but also a private one. “Certainly fashion and clothing are how we present ourselves to the social world and how we’re read by others, but it’s also very much about how we read ourselves,” Ben Barry, associate professor of equity, diversity, and inclusion at the Ryerson School of Fashion, says. There’s this intimate connection we have with clothes — it links how we feel in our bodies and who we feel we are inside.

“Clothing makes you aware of the edges and boundaries and borders of your body,” Barry says. “So wearing a dress, wearing women’s garments, even in the privacy of your own home, connects you to your body in a way that could make you feel comfortable or uncomfortable with how you perceive yourself.”

Especially if men are used to wearing pants and T-shirts all the time, a dress would display their bodies in completely unfamiliar ways. If they’re in spaghetti straps, their shoulders would be exposed in a way they have never seen before; if there’s a deep neckline, their chest would be cut differently; if there’s a short hem, their legs would take on a new shape.

“So walking around one’s apartment in a dress, how does that then make a man feel in his body? What does it make him think about how he has perceived his understanding of his own gender?” Barry asks.

Because of this, throwing on a dress isn’t quite so simple. “Wearing a dress in one’s home can make a man feel vulnerable in ways he’s never felt before, and that can trigger fear. Fear that you might not be so rigidly masculine as you’ve always thought.” It’s an identity crisis.

But if you take a moment and quiz the men in your life why they hadn’t skipped the shorts and put on a summer dress on a hot summer day, or went for the crumpled dress on the floor instead of their briefs, the majority of them will look at you like you asked what they would do if the world turned out to be flat tomorrow. It’s just not something they had ever considered. And there’s a reason for that.

Since we were little, each of us has been socialized into our binary identity, and we learned that there are specific rules for each gender — not only what they are, but also how they’re enforced and how they work.

“Kids learn all the same rules, but what they also learn is that the consequences are different for each of them, whether they’re a boy or a girl,” Dr. Paoletti explains. “The girls learn that some of the boy things they do they will get praised for, like being good at sports. But they’ll also get a lot of attention for being good at girl things — they get rewarded for being flexible. But boys learn that the girl rules are forbidden territory. If you trespass over there you’ll get smacked down, sometimes literally.”

Once they learn the rules they begin policing each other, where girls won’t let boys play with their dolls, or boys tease each other for liking girly things. “For the boys all the same rules, standards, and symbols are well understood, it’s just the feminine ones have all of these red flags. And I think it takes a lot of self-awareness and self-confidence to go against that kind of training.”

“Boys learn that the girl rules are forbidden territory. If you trespass over there you’ll get smacked down, sometimes literally.”

Of course, not many men actively think about this when they reach for pants on the floor. This is all going on in the background, and it comes out instead as a general feeling of “I really shouldn’t” when their training leads them to skip the dress that’s closer at hand. Paoletti likens it to the “step on a crack, break your mother’s back” game.

“I remember as I got older — and I no longer believed that if I stepped on a crack I would break my mother’s back — stepping on a crack still seemed like a really willful horrible thing to do. Even when the rule doesn’t make any sense anymore you still feel like you shouldn’t do that. You feel guilty about it.” This translates back to skirts, and how allowing yourself to put one on gives you this uncomfortable pang of knowing you shouldn’t like it, even though you don’t necessarily know why.

The straight men who do go for slip dresses and front-wrap skirts seem to have one common theme between them: They have fully rejected society’s binary, and the prejudices that come with it. And not in an “I walked in the Women’s March and signed a petition for gay rights” kind of way. They live it.

They were able to jump the hurdle of no longer seeing the clothes linked to their sexist and homophobic connotations, and just seeing them as another thing to slip into. And because of it, they don’t necessarily see their outfit choices as a political statement, but just an everyday outfit. Much in the same way a woman sometimes walks up to her closet and skips over her jeans for a column dress, they do it for the sartorial feeling.

“Wearing a skirt to me is just like wearing a pair of pants — it makes no difference. If the outfit looks better with a skirt, then I’ll wear the skirt.”

Take Akwete Osoka for example, who identifies as straight and is the founder and model of MaleMadonna. He skips identifying as cis because he doesn’t believe in being limited by labels, and chooses to identify simply as himself, Akwete. “Wearing a skirt to me is just like wearing a pair of pants — it makes no difference. If the outfit looks better with a skirt, then I’ll wear the skirt.” But that doesn’t mean he doesn’t experience backlash for his blasé attitude towards his wardrobe.

“Other men stare at me with disgust, as if I’m less of a man, or unworthy of being a man,” Osoka shares. People’s default reaction is to judge and assume, and he experiences from both men and women long, confused stares, constant laughter, pointing, name calling, and even moments of people taking out their phones to snap photos of him.

On Instagram, he had to go so far as to write a post letting people know that he was not gay, was not questioning his sexuality, and was just — really, truly — wearing a skirt for not other reason than he liked it.

“My fashion sense is me expressing my individuality; it’s me exploring boundaries that average guys are scared to explore because of what the rest of society will label them,” Osoka shared in the post. “Society is dying to say I’m gay but I’m not. Society is dying to label me bi, transgender, etc., but I’m not. Society has a headache dealing with me because I won’t allow myself to be within a label; I won’t let society cage me in.”

Jordan Sellers, a tech consultant who identifies as straight and cis gender, experiences something similar. “We constantly seek to categorize and organize things into little boxes. We continually hamper creativity by looking for differences in everything and everyone, rather than similarities.” And with these differences comes a need to dole out backlash.

A post shared by Jordan / JFS aka Playboi Jordi (@jordsels) on Sep 25, 2017 at 4:18pm PDT

When he was at a wedding in Charleston, South Carolina, he wore a skirt to the reception, and a man driving past shouted the word “faggot” out the window in anger. But to Sellers, that was a reminder of how much more worse it could be. “It was a reminder of my privilege, and stark contrast to what happens every day to queer people and POC. I mean, the prejudice was palpable in that city. I can’t even imagine being a black gay man in the south,” he shares.

The same thing happens in the LGBTQ+ community, where someone’s clothing choice is automatically linked to their sexuality, rather than allowing it to be a standalone fashion choice. Sean Santiago, editor and creative director of Cakeboy, an LGBTQ+ print and digital platform that turns a critical eye on gender and style, has made dresses and skirts part of his wardrobe and finds his wardrobe choices constantly combed for a deeper meaning. “Am I a crossdresser, am I doing this as a sex thing, am I getting off on these clothes? We just automatically jump there. If men are playing around with gender in that way, it becomes fetishized. It’s either about sexuality or part of some perversion.”

We’ve been grappling with these same truths for decades now, from a mother worrying about her son’s growing dress collection in the mid-century to headlines today questioning any man’s sexuality who decides to give a tunic a try. And no matter how progressive we think we are now, these same attitudes still persist in the same way they did a hundred years back.

As the advice columnist from 1938 pointed out, “We would send a man who paraded the streets in a decollete gown and high-heeled pumps to an asylum for mental observation, whereas a girl who gets herself up like an imitation man goes scot free.” Not until men can just as freely put on chiffon dresses as women could put on trousers, can we say that we have figured it out.