Mr. Rangel is experiencing what may be the most difficult period in his four-decade Congressional career as he prepares for a trial by a House panel this fall on a variety of ethics accusations.

Despite Mr. Rangel’s contention that the truth about his conduct will be exposed during the trial, most voters in Manhattan have already made up their minds: 73 percent think there is a measure of truth in the charges against him.

Mr. Rangel is accused of accepting four rent-stabilized apartments in Harlem, failing to pay taxes on a beachfront villa in the Dominican Republic and misusing his House office to raise money for an education center being built in his honor. He has denied wrongdoing, and has vowed to fight the claims, despite pressure from colleagues and even President Obama to retire.

While Mr. Rangel says that constituents have pleaded with him to run, and that none of his rivals are qualified to replace him, the poll shows that 70 percent of Manhattan voters want him to end his Congressional career: 46 percent said he should give up his campaign and step down at the end of his term, and 24 percent said he should resign now. Twenty-three percent said he should seek a new term.

One poll respondent who does live in Mr. Rangel’s district, Jeanne Cain, 89, said that after a distinguished career, the representative had stepped over the line.