Here’s the thing: I HAVE to wear a fedora.

If I want to wear a hat, I don’t have a choice. I am neither a ranch hand nor a roadie for a hair metal band, so cowboy hats are out. A lifetime of hiding my deep lack of coordination by avoiding sports makes me feel fraudulent in a baseball cap. Fedoras are my only hat option. I don’t swing dance or hang around pool halls with my sleeves rolled up, and I don’t want to reflect retro tendencies while I bleat out slow jams on an acoustic guitar, but I’m a nerd and I love fedora hats—despite their flaws and pretension, despite what they’ve come to represent. It’s a complicated relationship.

I’ve tried to give them up. I stopped wearing a fedora around the time it became the uniform for men’s rights activists and general creepy internet “Nice Guys.” I didn’t want to be counted along with those jerks, even by accident. I didn’t want people to expect a #notallmen argument to spill out of my mouth at any moment. The fact that MRAs and their ilk were snapping the hats up meant they were everywhere in shops, usually dirt cheap, and the thrifty shopper in me was constantly averting my eyes trying to avoid the temptation—but I resisted. I let my simple black fedora (even I can tell that those patterned ones are an abomination) gather dust in my closet. I missed wearing it, though, especially in cold or rainy weather.

The fedora symbolizes the very worst in men these days — the bull-headed pining for a “simpler” time when women accepted men’s authority, the crybaby “what about my feelings” whine when faced with women’s justified rage. I could understand why feminist bloggers wanted to slap the fedora off of men’s heads. The hat seems almost magical, like Frosty’s—put it on a guy’s head and the asshole in him comes to life. Knock it off, and the dude would have to flee the daylight, back to comment threads where he could be safely ignored.

It’s a horrible thought that I might be mistaken for an MRA, one of those men who have confused misogyny with trying to find a middle ground. Plenty of nakedly violent men now identify as men’s rights activists, but the classic MRA wants you to know that he’s too smart to be sexist. He wants to hear all about women’s views on things, there are just a few things he wants women to understand first—so many, many things. The men’s rights activist doesn’t want to tell you to shut up, he just wants to be sure you also hear all about his motivating pain. And he wants to go first.

Equally terrible is the “Nice Guy.” Nice Guys aren’t like those other men, the backwards baseball cap-wearing soul-patched bros you wouldn’t accept an open drink from. They’re post-modern gentlemen in fancy hats. Tell them all of your problems and then date them, please. Here’s a tip, boys: self-identifying as a “nice guy” makes you sound like you’re overcompensating for something. You become the human equivalent of an “I don’t want to sound racist” statement made by an asshole at a party.

Nice Guys seem to view the fedora as an opportunity for real-life retro cosplay, a chance to walk around in the skin of their flawed role models. They miss the good old days, back when sexism was cute. Women used to giggle when you called them “broads” or “tomatoes”—who wouldn’t want to be called a “tomato”? They’re delicious. In the past, or at least in the movies, women knew their place—and that can be a comfort for men in these modern times, when feminists want to do more than faint, cry, randomly disrobe, and fall in love with the leading man despite the fact he may present as a potentially dangerous sociopath. Fedora dudes are nostalgic for a fictional time when fedora dudes always got the last word. Indiana Jones and Philip Marlowe never got friendzoned.

Eventually, though, I realized that while my anti-fedora-dude stance was strong, my anti-fedora stance was actually very silly. A fedora isn’t a membership badge for some retro-sexist club, and it’s not a filter to keep modern ideas from seeping into your brain. It’s just a hat. A friend of mine reminded me that in order to be mistaken for a MRA I’d actually have to start talking like one, and as there was no chance of that happening, I might as well get over myself and just wear the damn hat if I wanted to.

Still, in order to wear one again I wanted to reach a compromise with myself. If I was going to wear something so rich with sexist symbolism, I had to earn it.

I started allowing myself 10 minutes of fedora-wearing time for every pro-feminist argument I made in public. I decided that credits are only earned from conversations that I have with men, because any conversations I had with women about feminism should be more about listening.

Points stacked up fast. I didn’t heave to engage in knock-down, drag-out fights, either—something as simple as mentioning that misandry doesn’t constitute a hate crime, or pointing out that Steven Moffatt’s writing on Doctor Who is wicked sexist, can do the trick. By the end of the first week, I had enough time saved up for dinner and a movie out with my hat.

Doing this makes me feel a little better about the fedora-related side eye that I might be getting—but it does much more than that. The more I used the system to earn those precious minutes, the more I realized how lazy a feminist I’d become in the last few years. I’d started letting an eye-roll take the place of any kind of vocal dissent. I had reasoned that since these dudes were never going to change anyway, there was never any point in arguing with them. Earning hat time made me remember that there’s always a point in arguing, even gently, with sexist men.

As I tried to be more vocal about my beliefs, I remembered that nerds make great feminists because we’re born debaters. We can advance the struggle greatly if we all just contribute a quarter of the energy we usually burn arguing about Earth 2 Batman or Storm’s mohawk towards a #YesAllWomen argument.

I want to take the fedora back, because it’s fun to tell MRAs that they can’t have something. They get so mad. But I don’t just want to take it back—I want to earn it. It’s an opportunity to make a small contribution, one argument at a time. Maybe if other nerds and lazy dude feminists join in, the MRAs will turn away from the fedora. They might move on to other sartorial choices that I don’t care about—pipes and bowler hats might suddenly start looking more attractive to them. Feminist men who love smoking jackets: start polishing your arguments now.