If we’re ever going to reach gender equality, it may have to be white men who lead the charge. New research shows that managers are actively penalized for promoting diversity in the workforce, regardless of gender or race—unless they are white men.

To find out why 85% of C-level leadership positions at top companies are filled by white men, and why that number seems to be holding, David Hekman from the University of Colorado Boulder led a team of management researchers in a study that evaluated over 350 executives on how well they tried to improve the diversity of their teams.

In their study, published in the Academy of Management Journal (subscription), Hekman and his team evaluated the managers on their “diversity-valuing behaviors,” like hiring women and minorities, and respecting other cultures, religions, genders, based on peer reviews of their subjects.

“Much to our surprise,” Hekman and the study’s second author Stefanie Johnson write in the Harvard Business Review, they found two startling discoveries: First, no one gets a pat on the back for promoting diversity at work. No one, regardless of race or gender, was evaluated more positively by their bosses for advocating for more diversity in the workplace. Second, however, women and non-white executives were judged more harshly by their bosses when they did engage in “diversity-valuing” behaviors in the workplace.

Which means that the only group that isn’t punished in some way for advancing diversity in the workplace is white men.

This does not bode well for many people leading diversity initiatives. If women and people of color are vocal about these kind of issues at work, they may be second-guessed or criticized—because doing so acknowledges their low-level status, as Hekman and Johnson explain in HBR.

“We argue that diversity-valuing women and non-whites are rated lower than their non-diversity-valuing counterparts because diversity-valuing behavior activates subtle and unconscious stereotypes about women and non-whites as being less competent,” the authors write in their study.

Those stereotypes may then influence how managers review their employees’ performance and competence. On the other hand, white men enjoy a higher status, and aren’t viewed any less favorably for wanting to lend a hand to other racial or minority groups.

The results of Hekman’s second study, in which about 300 subjects looked at photos of fake hiring managers and evaluated them based on their hiring practices, were just as depressing: white manager saw “no effect on performance ratings” when they promoted diversity, the authors write. On the other hand, “among the non-white managers, diversity-valuing behavior had a negative effect on performance ratings.”

The implication, the authors say, is that when women and minorities become high-level leaders, they may be less inclined to help other women or minorities advance their careers. The onus, then, is on managers and executives of all backgrounds to be aware of this bias and override it.