When I began co-hosting a video series on Game of Thrones, I braced myself for complaints about my voice. Not because it’s a bad voice—I have a relatively nondescript, medium-toned voice—but because I had heard from countless women in the radio and podcasting world that no matter what you sound like, as long as you are a female speaking to a general-interest audience, you will be told that you have an irritating voice. This is doubly true if you work alongside a man.

After all, a woman is talking, so she must be doing something wrong.

I was almost looking forward to it, honestly. What about my voice, I wondered, would some snippy listener zero in on? It didn’t take long to find out. “Amanda should pronounce her ‘t’s’,” chimed in an irate commenter, zeroing in on the slight West Texas drawl that lingers in my voice, despite years of living elsewhere.

But it’s not about a particular accent—Katie Mingle, the producer for the popular 99% Invisible podcast, gets so many complaints about her female reporters’ voices that she crafted an auto-reply email to handle the volume of people writing in to grumble.

Radio shero @katiemingle of 99% Invisible crafted a lovely auto reply for listeners complaining about women's voices pic.twitter.com/fLoVu5aTte — Jenna Weiss-Berman (@WBJenna) July 11, 2015

In a January segment for the show, This American Life also reported having a very similar level of complaints—but again, only about the women. A number of female reporters, in their perfectly wonderful NPR-style voices, explained that they had to endure a constant barrage of complaints about how they speak.

This week, NPR also reported on the way that women are always, especially if they work in audio formats, are being told the way they talk is all wrong—from upspeak, where people lilt their voice for emphasis, to the dreaded vocal fry, where people drop and rattle their voice slightly for emphasis. You’ve heard it before: Vocal fry is that thing where you drag your voice down a little and creak it for emphasis. (Imagine a young woman saying “sooooo cuuuute” with a little growl and you got it.)

I had joined a peculiar sisterhood, one where no matter how high or low or mellow or sharp your voice is, if you are female and doing voice-only work, some audience members will take the time to explain that you have a terrible voice and need to shut up posthaste.

Is it sexism? Should you accuse a person who complains of a woman’s voice of sexism, he or she will immediately deny the accusation. Certainly (or should I say “cerainly”?), that’s my experience. But to people who make these complaints, it’s not that they have ingrained prejudice against women, it’s that women talk all wrong. If they would just learn to speak correctly, the sexists say, then it wouldn’t be a problem.

A woman is talking, so she must be doing something wrong.

Take, for instance, a recent battle at Slate between writer Bob Garfield and some of the more feminist-minded staff members. Garfield, released an epic rant on the Lexicon Valley podcast about vocal fry, which he deemed a young, female affectation that he wants to scrub from lady voices across the land. He got immediate pushback from a number of women, including Slate’s Amanda Hess, who pointed out that for some reason, it’s always women who get this kind of nitpicking abuse of the way they inflect their voices.

Garfield went on the defensive, claiming that his critics were accusing him of hating women. “All affectations are annoying,” he clarified. But calling vocal fry an affectation, like a fake accent, is missing the point. It’s used almost exclusively for emphasis, and most women who do it don’t even realize they’re doing it. Without the ability to stress certain words or phrases, women’s voices will be softer, less assertive, and less interesting. Easier, in other words, to ignore.

This how sexism tends to work. Very rarely does someone just come right out and say that women should shut up and talk less. Instead, they come up with a lot of irrelevant reasons for women to feel self-conscious about their voices. Most people who lodge these complaints likely don’t mean to bully women into shutting up. It’s often a sense of vague irritation, which they pin on the women themselves, instead of wondering if it’s just that they themselves have been trained to resent it when women are the ones holding the floor.

The current trend of policing women for vocal fry—which you can see in Time, the Atlantic, and Today—shows exactly how true that is. For years, the main complaint about women’s voices was that they are too high-pitched and that women use upspeak, which is a way of inflecting your voice so it goes up higher at the end of sentences, making it sound like a question. This was often derided as “Valley girl” speech, and mocking it became a huge cultural trend—from the song “Valley Girl” to the movie Clueless.

But vocal fry is the opposite of upspeak. You’re pitching your voice lower, not higher. You don’t sound like you’re asking a question but like you’re making a definitive statement. “So we’re wrong when we raise our voices, and we’re wrong when we lower them,” Hess pointed out. It’s hard to avoid the conclusion that the only “right” way for women to speak is “not at all.”

This how sexism tends to work. Very rarely does someone just come right out and say that women should shut up and talk less.

For women, this is a game that’s unwinnable, no matter how powerful you are or what stage you’re on. Hillary Clinton’s speech has long been chided by critics, with Fox News’ Frank Luntz and Sean Hannity arguing that her voice “turns people off” because “they feel like they’re being lectured.” Arguing that it “just sticks in your ear like an ice pick,” Glenn Beck added that her voice “makes you envy the deaf” and “makes angels cry.” And former New York Times editor Jill Abramson’s voice was once referred to as “the equivalent of a nasal car honk.” And that was in the New Yorker.

We also know that the relentless attacks on women’s voices—particularly vocal fry—are sexist for one very simple reason: Men are rewarded for using the same vocal tools.

As the hosts on This American Life pointed out, Ira Glass can hardly go a sentence without frying his voice, which is why he sounds like he’s speaking ironically all the time. However, they don’t get a lot of letters about Glass’s voice on the show, and his manner of speaking is actually part of his popularity. According to the Salt-Lake Tribune, Glass’s “distinctive” speech makes him “one of the most recognizable voices in radio today.”

https://www.youtube.com/watch?v=8X7Ni_TnZEg

Ira Glass isn’t the only man that relies on vocal fry. Radio producer Ann Hepperman put together a fun mix of men from various audio programs frying their voice, and the second you hear it, you realize that this inflection—or “affectation,” if you’d rather—is just a very normal way to emphasize a point or to inject a little sarcasm or humor into your sentence.

In fact, it becomes all too easy to think of all the men who do it—from Johnny Depp to Bruce Willis and Vin Diesel. Owen Wilson can hardly open his mouth without it.

We also know that the relentless attacks on women’s voices are sexist for one very simple reason: Men are rewarded for using the same vocal tools.

“People are busy policing women’s language and nobody is policing older or younger men’s language,” Stanford linguistics professor Penny Eckert told NPR. That it’s fine when men use vocal fry but irritating when women do it says a lot about what many people don’t want to hear from women: humor, authority, and sarcasm.

Sometimes women are undermined by attacking their voices. Sometimes they are undermined, as Amy Schumer brilliantly pointed out in her “12 Angry Men” parody, by declaring they aren’t attractive enough. Sometimes women are simply told they aren’t funny, just another way to argue women have nothing to say. In 2008, Christopher Hitchens offered the most famous example in a much-criticized Vanity Fair essay on women’s humorlessness, but as a recent gaffe from former Disney CEO Michael Eisner showed, complaining about women never goes away.

We like to think these are all separate incidents, but they are part of a larger pattern, a constant devaluing of women’s worth—or of their need to speak up and be heard. Instead of learning to listen, it’s a whole lot easier to tell women to stop talking.

Amanda Marcotte is a Brooklyn-based freelance journalist who writes for Rolling Stone, Slate, and USA Today.

Photo via MCAD Library/Flickr (CC BY 2.0)