Washington (CNN) Retiring Rep. Ryan Costello criticized President Donald Trump on Wednesday for chastising Republicans who didn't support him and had lost their midterm elections during his freewheeling news conference Wednesday.

"It's like, to your point, dancing on somebody's grave," the Pennsylvania Republican told CNN's Kate Bolduan on "Erin Burnett OutFront." "It's highly inappropriate, and it's deeply offensive, and every single one of those members took tough votes in order to advance a center-right agenda because they believed in it, but they took a lot of heat for it and the President should be thanking them for putting up the tough votes and advancing an agenda which I believe we are better off economically because of a lot of the policies that have been implemented."

"And he didn't take that opportunity to do that," Costello continued. "Instead, he decided to say negative things about them. I just think it's so entirely wrong. It's a failure of leadership."

At his news conference Wednesday, Trump mocked the Republicans who had lost their elections and "didn't want to have any embrace" of him during their campaigns.

"You had some that decided to, 'Let's stay away. Let's stay away.' They did very poorly," he said. "I'm not sure that I should be happy or sad, but I feel just fine about it."

Read More