The #MeToo movement has brought unprecedented attention to sexual harassment and assault. It’s revealed just how many women feel besieged by sexually predatory behavior—especially in the workplace. The wave of women coming forward has shown that sexual harassment is the rule in many institutions.

And #MeToo has only revealed a small piece of a much larger problem. Although the most high-profile #MeToo stories have focused on celebrities or executives, most victims are disproportionately young, low-income, and minority women. Also less evident in the #MeToo movement have been cases of sexual violence: where shaming, trolling, threats, and unwelcome advances have given way to rape, physical violence, and even forms of torture—of which choking is the most common.

In its most extreme cases, it can literally be a matter of life and death, and yet sexual harassment and violence remain largely hidden by an elaborate system of denial, gaslighting, and retraction of accusations by women. Meanwhile, unrepentant abusers are often comforted or excused while victims are blamed. How did we get here? Moral philosopher Kate Manne’s book, Down Girl: The Logic of Misogyny, helps explain. Thanks to Manne, the undue comfort that men receive now has a name: It’s called himpathy. And, together with how she defines misogyny, Manne provides a useful framework for understanding not just the present #MeToo moment, but what came before. For Manne, misogyny is not simply “men who hate women.” That’s far too simplistic, she says. Rather, it’s a far-reaching, punitive social system that keeps women in their place by rewarding compliance and punishing resistance to the gendered social order. This disciplining role of misogyny has escaped attention for a variety of reasons, chiefly, the social shield of himpathy.

Himpathy, a term destined to become part of the feminist vocabulary, names a problem previously unrecognized—and perhaps that’s the first step in solving it. Manne defines himpathy as the “excessive sympathy sometimes shown to male perpetrators of sexual violence,” in the attempt to preserve their reputation, power, or status. Accused men, especially those with privilege, are broadly treated with deference by the media and the public, and if they’re brought to court are given lenient sentences. This is so common as to be a given for men in power. Harvey Weinstein is a case in point. Wielding control over the film careers of many and trading on his artistic reputation, he escaped unscathed for decades. Excuses are abundantly generated: alcohol, flirtation taken too far, or provocation on the part of the victim. Himpathy builds on the idea that sexual predators and rapists are creepy monsters, not “golden boys.” Correspondingly, the women in these situations are characterized as hysterical, misguided, or liars who misread the intentions of their attackers. Himpathy is a helpful explanation of the response after sexual abuse allegations are revealed. Over and over, we’ve seen victim blaming and rewriting of the story by friends, family, media, and sometimes even the victim. Responses to #MeToo revelations by close-at-hand onlookers are often characterized by shock and guilt for having looked the other way when powerful and respected men are involved. But himpathy is certainly not a recent phenomenon. Historically, misogyny and himpathy have been normal, if unrecognized, fare for women in the workplace. Sexual coercion at work had to be named before it could be fought, and feminists of the 1970s identified common experiences women suffered by naming marital rape and domestic abuse. The term “sexual harassment” in the workplace was defined by Lin Farley in her 1978 book, Sexual Shakedown: The Sexual Harassment of Women on the Job, as “unsolicited nonreciprocal male behavior that asserts a woman’s sex role over her function as a worker.” Farley joined the legal scholar Catharine A. MacKinnon in pressing the courts to consider it part of “sex discrimination” under Title VII of the Civil Rights Act of 1964. The Act gave women and minorities new rights in employment. But there was still backlash. A law on the books is only the first step in triggering a cultural shift. And law is not useful unless some are willing to use it and make a claim.

The recognition of sexual harassment as a form of employment-related discrimination opened the floodgates: The Equal Employment Opportunity Commission began receiving tens of thousands of claims each year. Even with a rush of claims, many from low-wage workers, the definition of sexual harassment as interpreted by the courts is narrow and fails to consider the disadvantaged social circumstances of women that dissuade many from seeking legal recourse. Over the next 40 years, as women entered previously male-dominated fields, sexual harassment, though illegal under the law, persisted. Take, for example, the high-profile cases of Anita Hill and Clarence Thomas in 1991 or Bill Clinton and Paula Jones in 1994. Despite attracting a great deal of attention, these failed to mobilize a mass movement. In both cases, the men involved were held by many to be blameless while Hill and Jones were scrutinized for ill intentions. Hill’s accusation on national television ultimately did not stop the Thomas confirmation, and Jones faded into obscurity. High-profile cases like these are easily dismissed as aberrations, a moral failure of one individual, a political plot, or gold-digging on the part of victims. Non-transgressing men benefit from a system that keeps women in their place, and low-profile cases continue to be invisible. The backlash against #MeToo, in an already global movement, has begun. Sometimes the case is taken up by women, such as the actress Catherine Deneuve, who evoked the French tradition of seduction against sexual puritanism: “Clumsy flirting is not a crime,” she said. Claire Berlinski, writing for The American Interest, charged that in #MeToo, “mass hysteria had set in [as] a form of moral panic” that misinterprets naturally romantic interactions as nefarious.

This women-against-women narrative is part of the story of misogyny and himpathy—and it’s part of why it’s so difficult to remedy. By standing by their man, “good women” show their deference and act as enforcers. In exchange for upholding gender norms—and participating in misogyny by punishing those who don’t—they earn favors and advancement, which reinforces even further the social deviance of the victims. After all, women can say no, these defenders say. But if you are not a woman with executive power or Meryl Streep, saying no is difficult. Women who work to support their families have few options. When the choice is between your job and your dignity, himpathy is likely to work as a silencing mechanism. Unless #MeToo successfully expands beyond professional women by reaching out to empower pink- and blue-collar women who suffer in silence under male supervisors, it will leave its mark but will not have done its most significant work.