(CNN) The House ethics committee launched last week an investigation into Michigan Democratic Rep. John Conyers after allegations surfaced from multiple women that he sexually harassed them.

But the panel -- charged with policing wrongdoing and disciplining lawmakers -- has a thin record of acting on claims of improper behavior, and the bulk of cases that come before the committee languish for months, and sometimes years.

House Minority Leader Nancy Pelosi, a former member of the committee, sent a letter urging quick action this week on the Conyers matter.

"The Committee on Ethics has a great responsibility to proceed expeditiously as well as fairly into any investigation of credible harassment and discrimination allegations," Pelosi wrote, adding that if more resources were needed the committee should alert leadership.

California Democratic Rep. Jackie Speier told reporters that she thought the ethics committee should be able to complete its review "within weeks."

But most lawmakers don't believe that is realistic.

"The reason being is you can use it as a political tool if you choose to and I think that's exactly opposite of how the ethics committee should operate. It takes a long time and if you think they are going to do an investigation of Mr. Conyers in a week good luck on that," Utah GOP Rep. Chris Stewart told CNN.

He pointed to the probe of House Intelligence Committee Chairman Devin Nunes, on charges that he disclosed classified information. Stewart argued it could have been resolved in a week, but has dragged on since April.

Missouri Democratic Rep. Emanuel Cleaver served on a bipartisan ethics working group that screened matters and said it's not doable to wrap up these types of probes quickly. He defended the process of having an equal number of Republicans and Democrats sifting through the evidence, saying it's important to be thorough and members take the responsibility very seriously.

"I've sat around that table listening to evidence. For me it was the most bipartisan experience I've had on Capitol Hill," Cleaver said, adding, "I never saw any intentional stalling."

By design the ethics committee isn't the place on Capitol Hill to adjudicate cases involving employer-employee relationships. The Office of Compliance was set up in 1995 through the Congressional Accountability Act, after lawmakers and aides decided that the 535 separate offices of members acted like their own small businesses. Without a human resources department to turn to in instances where an employee felt a boss or a colleague mistreated them the goal was to make that office a resource, and responsible for determining if any action was needed.

But there's little evidence that many aides feel that is a good place to go, or believe that the ethics committee is either.

Meredith McGehee, now executive director of Issue One, a nonprofit advocacy group pressing for government reforms, was part of the lobbying effort to establish the Office of Compliance. She said there are "serious flaws" in how the current process works, and going to OOC or the ethics committee to report a Member of Congress for harassment is viewed as a last resort.

"If you are branded as disloyal then you are making a decision to find another line of work," McGehee said.

One former aide to Conyers, Deanna Maher, told CNN that Conyers made three sexual advances toward her when she worked for him in his district office in Detroit from 1997 to 2005. She reported him to the ethics committee in January 2004 for other issues related to improper management in the office and reported a staffer in the office for sexual harassment, but there is no evidence that the committee acted on any of those allegations.

Rep. Susan Brooks, R-Indiana, who chairs the ethics panel, admitted in an interview on C-SPAN earlier this month that when Speier testified that there were two current House members who she was told had engaged in improper conduct, Brooks didn't know who they were.

"When Jackie mentioned that there were two members, and she actually went into some detail about the allegations, I am not familiar, as chair of the house ethics Committee, I am not aware of which two members she is referring to," Brooks told C-SPAN.

The ethics committee tends to focus on instances of allegations of criminal wrongdoing, or potential violation of rules governing the offices not using official staff or resources for political purposes, but even in those cases the proceedings routinely last months, if not years.

When sexually explicit communications surfaced between Florida Republican Rep. Mark Foley and teenage male House pages in 2006, the ethics committee did launch a major probe, and called in top GOP leaders, including then House Speaker Dennis Hastert , to determine whether there was any coverup of the matter. Foley resigned, and the committee later issued a report that criticized top Republican leaders for failing to act on warnings, but it did not recommend any sanctions.

The panel is currently still investigating another matter regarding Conyers, for improper use of his office payroll. The Office of Congressional Ethics, the outside independent entity that screens matters found that "there is substantial reason to believe that Representative Conyers provided a member of his congressional staff with compensation that was not commensurate with the work she performed, in violation House rules and standards of conduct."

The committee released the report from OCE in August and opened a probe then, but it's unclear whether there has been any movement on the issue because of the committee's rules not to discuss any open matters.

OCE addressed allegations of employees reporting improper behavior in a report on North Carolina GOP Rep. Mark Meadows' former chief of staff, Kenny West. Witnesses interviewed by OCE said that aides in the DC office said they "were made to feel uncomfortable" by West. There is no indication they ever reported the instances to OOC and some of the aides who spoke to OCE ended up leaving the office. The matter was referred to the ethics committee in May 2016, but since West is no longer employed in the House it's unclear there will be any follow up.

The white hot environment where sexual harassment allegations against media figures, Hollywood executives and corporate chiefs are followed by swift action, but little repercussions for similar behavior on Capitol Hill is only increasing the perception that Congress is using a double standard.

"We need to make sure that this time, this is a different climate and we need to see real change and everybody in this chamber needs to understand that we've got to become more transparent and do some changing of our processes," Michigan Democratic Rep. Debbie Dingell told CNN.

There are cases where the ethics committee can act swiftly. In instances where lawmakers were arrested during political protests, the committee acted in a matter of days, quickly deciding that the Congressman paid their fine and end the investigation.

McGehee agrees that the public pressure may make the Conyers' latest ethics case one that could end up moving through faster than most matters before the panel.

"In this case, I'm sure they will because the political imperative to do so is so strong, unlike the many other instances when the political imperative is to drag it out," McGehee told CNN.