WASHINGTON—A House ethics panel sitting in judgment of Rep. Charles Rangel agreed Monday to all of the prosecution evidence against him, despite his attempt to delay the case by walking out for lack of a lawyer.

In retreating from the trial on its first day, the New York Democrat said he could no longer afford a lawyer and that it was unfair to make him proceed without one.

The...