Of all the female celebrities to eschew the feminist label, Susan Sarandon seemed a most unlikely candidate. The 66-year-old actress has long been outspoken about everything from women’s reproductive rights to voting and human-rights issues.

But when The Guardian asked in a recent interview whether she would call herself the F word, Sarandon turned up her nose.

“It’s a bit of an old-fashioned word,” she said, referring to herself as a “humanist” instead because that word is “less alienating to people who think of feminism as a load of strident bitches and because you want everyone to have equal pay, equal rights, education, and health care.” The feminist identity is used more “to minimize you,” she said, adding that her 28-year-old daughter “doesn’t even relate” to the term.

In other words, Sarandon’s all for gender equality, but she doesn’t call herself a feminist because she thinks the label is too stigmatized to be taken seriously anymore. That could be hard swallow for feminist icons like Gloria Steinem, who have rallied for women’s rights alongside the actress, or for her feminist fans, who watched her endorse the “White Man’s Dick” as a symbol of citizen status in a mock PSA to raise awareness for UniteWomen.org. But Sarandon has been met with far less derision in the mainstream media than Taylor Swift or Carla Bruni-Sarkozy or Katy Perry, all of whom have said they want nothing to do with the label.

Perhaps that’s because, despite Sarandon’s belief that the label no longer applies, she has fought the fight. Her disassociation with the feminist label and its negative connotations just boils down to semantics. The ideology hasn’t lost its value; it’s just been poorly branded.

At least that’s what some experts think.

“The word ‘feminism’ is used consistently to demean or dismiss people’s ideas,” says Sarah Sobieraj, a professor of sociology at Tufts University who has written on the subject. This demeaning comes mostly from feminism’s enemies, who perpetuate stereotypes associated with the movement. And yet as Sobieraj points out, “It’s the identity that’s being used to silence someone or to dismiss them more often than the actual idea or policy or assertion.”

Indeed, some powerful women—yes, that includes Katy Perry and her frosting-projecting breasts—say they are uncomfortable with the label even though they embrace feminist ideals. Last year, Yahoo CEO Marissa Mayer said that while she “certainly believes in equal rights” and that “women are just as capable,” she thinks feminism has become a “more negative word.”

But feminist writers seem tired of women misunderstanding their movement. Take Salon’s Mary Elizabeth Williams, for example, who bluntly addressed Perry’s and Mayer’s remarks: “Let me just point out that if you believe in the strength of women, Ms. Perry, or their equality, Ms. Mayer, you’re soaking in feminism.” In response to Carla Bruni-Sarkozy’s dismissal of the term, Williams simply laughed on the page: “Ha! HA HA HA!” When Taylor Swift answered the predictable “do-you-consider-yourself-a-feminist question” in the negative (“I don’t really think of things as guys versus girls”), Jezebel slipped into Mean Girl mode: “It’s like she doesn’t understand what a feminist is. Is this what happens when you’re homeschooled after the age of 15?”

It’s understandably maddening for feminists when women agree with their views in one breath and ostracize them in the next. But the feminists who rail against these women aren’t helping their own cause. Perry and others certainly aren’t going to want to identify with the movement more if feminists laugh at them or dismiss them as uneducated fools.

And plenty of feminists wear the label with pride but still feel the need to add a tagline to that label so their voices aren’t dismissed. Even Jessica Valenti, co-founder of Feministing.com and one of today’s best-known feminist bloggers, identifies herself at The Nation with a caveat: “Feminism, sexuality & social justice. With a sense of humor.”

That caveat is a strategy to be taken seriously, says Sobieraj. “The subtext is: please listen to what I have to say because I’m not the uptight, man-hating feminist but the funny kind.”

Whether you’re a celebrity like Sarandon or an ordinary citizen, believing in or fighting for women’s rights doesn’t fit neatly into the feminism box, at least as far as semantics go. But for many women, it’s still relevant, and it’s worth asking others, including men, how they feel about the F word. Perhaps then we’ll better understand why it carries such pernicious stereotypes—and how to effectively rebrand it.