Embattled U.S. Rep. John Conyers has no plans to resign amid allegations that the congressman sexually harassed several women when they worked on his staff, his lawyer said Wednesday.

Detroit-area attorney Arnold Reed told The Associated Press that the 88-year-old Conyers is going to fight claims that he inappropriately touched the women and that anyone making the allegations should be prepared to back them up.

Reed said Conyers is "innocent and will cooperate with any investigation that ensues."

Get Breaking News Delivered to Your Inbox

"He's going to fight these allegations tooth and nail if he has to with evidence, with documentation, witnesses, whatever he has to do," Reed said. "And the accusers will have to prove up their case."

Last week, BuzzFeed News reported that Conyers had settled a complaint in 2015 from a woman on his staff who alleged she was fired because she rejected his sexual advances.

BuzzFeed reported that Conyers' office paid the woman more than $27,000 in the confidential settlement. BuzzFeed also published affidavits from former staff members who said they had witnessed Conyers touching female staffers inappropriately or requesting sexual favors.

A former scheduler also filed a complaint earlier this year, but later dropped it. The Associated Press hasn't released her name. And a third ex-staffer, Deanna Maher, said Tuesday that in 1997 Conyers undressed to his underwear in front of her and twice touched her leg inappropriately.

Reed said he met with Conyers, who was at his Detroit home Wednesday and mostly remained indoors before being driven away in the afternoon as reporters gathered outside the gated driveway.

Conyers' wife, Monica Conyers, told reporters that her husband is entitled to have the judicial process play out "before we start being his judge and jury ... and tarnish all of these years of his legacy for nothing."

Conyers first was elected to the U.S. House in 1964. He missed two roll-call votes late Tuesday and was photographed by a passenger boarding a flight to Detroit from Washington.

"It's very unfortunate to see him fight so long for so many people and to automatically have the allegations assumed to be true," said his son, John Conyers III.

He noted, however: "And of course, with sexual assault, women are to be believed. But in this instance he has no history of this."

Conyers told Reed that fears for his family over media coverage prompted his return to Detroit late Tuesday from Washington. Reed said the congressman will return to Washington when he feels it's safe to go back.

The House Ethics Committee is investigating Conyers, who announced Sunday that he would step aside as the top Democrat on the Judiciary Committee.

House Minority Leader Nancy Pelosi pressured Conyers behind the scenes to leave the chamber, according to a senior House aide, who spoke late Tuesday on condition of anonymity to discuss private conversations.

Also Tuesday, members of the Congressional Black Caucus met with Conyers and explained to one of their founding members why he should resign, but stressed the decision was up to him, the senior House aide said.

On Wednesday, Rep. Kathleen Rice walked out of a Democratic Caucus meeting, saying the issue wasn't being taken seriously.

Rice said Pelosi spoke only "very briefly" about sexual harassment and focused mostly on efforts by female members to combat it.

"But let's talk about the big elephant in the room. That's why I was done with the meeting," Rice said.

"I don't have time for conversations that are not real, that are not going to advance the ball for all of us here in Washington, and I for one am not going to stand silent even in the face of pressure from leadership," she added.

Caucus chairman Rep. Joseph Crowley said at a news conference after the meeting that Caucus members "believe these are very, very, very serious allegations."

But, he said, calling "for a resignation does not actually create the resignation" and that an ethics committee review could "bring this to the forefront."

Three Democrats have suggested Conyers should resign: Rice, Rep. Earl Blumenhauer and Rep. Pramila Jayapal.

Monica Conyers said her husband was spending time Wednesday with family and that she was sure he would return to Washington, eventually.

"I don't know when," she said. "He's doing good. Right now, he's just hanging out with his family."

Conyers is among a number of prominent men in politics, entertainment and journalism who have been accused of sexual misconduct in the wake of explosive allegations against former Hollywood executive Harvey Weinstein.

Associated Press reporter Alan Fram in Washington, D.C., contributed to this story.