(CNN) As more and more women come forward with sexual harassment allegations in the workplace, others are remaining silent -- not because they don't want to share their powerful stories, but simply because, legally, they can't.

These are the women who have signed non-disclosure agreements.

Instead, they are forced to watch as their peers courageously share instances of sexual harassment in the workplace, all because these women sought to bring justice to their harassers, and to do so, they compromise with silence.

On Capitol Hill, they are particularly restricted, several people told CNN.

"It's hard," said one woman who signed an NDA after making a sexual harassment claim to the Congressional Office of Compliance. CNN is withholding her name because she is not supposed to speak about her settlement.

"I understand that I signed the agreement and so that's the way it is, but I do feel as if my voice is missing from the chorus of women speaking now," she said. "You do feel silenced and that's something I've had to grapple with."

This woman made her claim in the last five years, well before the reckoning that has brought down men in powerful positions in Hollywood, media and politics.

"The climate back then was different than right now. Back then it seemed untenable," she said. "It just seemed like the best option for me at the time. ... But the positive is that I'm able to put that part of my life behind me without constantly having to revisit it."

In hers and many cases, part of the strict confidentiality provisions of the complaint process on Capitol Hill preclude her from revealing any information, or speaking disparagingly about the congressman she accused.

The Congressional Accountability Act, set up by Congress in 1995, was set up in part to deal with charges of sexual harassment, as well as other labor disputes within Congress, and does include some confidentiality provisions.

According to the statute within the law, as an accuser takes their complaint through the process within the Office of Compliance, there is strict confidentiality in the counseling process, strict confidentiality in the second step, the mediation stage, and confidentiality is a bit looser in the hearing stage.

Sources within OOC tell CNN that employees are able to waive confidentiality during the first counseling stage. But once the parties enter mediation there is confidentiality in terms of the materials produced within the mediation.

The non-disclosure agreements that many sign, however, are something separate, outside of the Office of Compliance's process.

"The NDAs at the end, that is something else that parties come up with themselves," Susan Tsui Grundmann, executive director of the OOC, told CNN. "We don't demand it. It's something that binds them. We don't control it."

However, this is often a gray area.

The office of Rep. Jackie Speier, who has been one of the leaders on Capitol Hill to overhaul the process, says it is their understanding from the mediators, lawyers and accusers they've spoken with that there are discrepancies between the letter of the 1995 law and what victims have been subjected to.

JUST WATCHED Dingell: Conyers accusations deeply disturbing Replay More Videos ... MUST WATCH Dingell: Conyers accusations deeply disturbing 01:28

Aides in Speier's office say they have been told by those who have been through the process that they must remain silent, and agree to the NDA in perpetuity, to progress and file a complaint.

"Taxpayers foot the bill and the harasser goes on with his or her life," Speier said during congressional hearings on the topic in November. "There is zero accountability and zero transparency. I might also add that during that process, the victim can't even communicate that they are going through an OOC process to their family, to their friends or to anyone in their religious community. So it's really no wonder that staffers do not seek this process at all."

This is one of the chief areas of focus for her legislation, proposed on Capitol Hill -- to spell out specifically that NDA agreements would be voluntary.

"It's extremely one-sided and heavy-handed, and basically imposes confidentiality and non-disclosure for life," said attorney Debra Katz, who has handled several Capitol Hill cases.

Contrast that to the accused, who can defend themselves, Katz said, referencing Michigan Democratic Rep. John Conyers, who denied wrongdoing after a BuzzFeed News report made public that he settled a wrongful dismissal complaint in 2015 after allegedly sexually harassing a staffer.

"I think that when you have an agreement that's so broad that says you cannot discuss -- with no carve out for accountants, advisers, therapists," Katz said. "People feel completely trapped and unable to move forward in their lives. NDAs don't allow them to heal. They can't talk about it and can't recover."

The women who often seek help from the OOC need paychecks and jobs to survive in Washington.

"And they make a calculation that it's not going to be good for their careers going forward to make a (public) claim, and they'd rather get something than nothing," Katz said.

The roughly $27,000 settlement revealed by Buzzfeed against Conyers is an average settlement for Congressional cases, Katz said, even though they are very low compared to the private sector.

"People get very lowballed in Congress," she said. "It's ridiculously low, it's paltry. Someone with these claims in the private sector, this would be a highly monied claim."