Story highlights Many Democratic congresswomen wore white to President Donald Trump's speech Tuesday

North Dakota's at-large congressman repeatedly criticized that decision

Washington (CNN) Republican Rep. Kevin Cramer defended his comment Wednesday that the Democratic women who wore white to President Donald Trump's joint address were "poorly" dressed, telling CNN that they looked "silly" and that he didn't buy their argument that it was done in honor of the suffrage movement.

The at-large congressman from North Dakota also reiterated that he hasn't ruled out a Senate bid next year for the seat currently held by Democratic Sen. Heidi Heitkamp, and he said Trump has already pledged his support should Cramer decide to run.

The women, who represented the House Democratic Women's Working Group, said they were wearing white not only in memory of the suffrage movement but also to show Trump their support for a number of issues affecting women, such as affordable health care, reproductive rights, equal pay and paid leave. The effort was also a nod to the start of Women's History Month, they said.

Cramer, however, said the women "were really there to be rude to Donald Trump."

"That was obvious, not just, not by their clothes, but in addition to their clothing, their gestures, their hand gestures, their thumbs down, their quick exit from the gallery ahead of the President," he said. "Their behavior in general."

Read More