(CNN) Former Vice President Joe Biden defended his embrace of bipartisanship in a divided era Thursday, responding to criticism he has received for praising Republican congressman during the midterms.

"I read in the New York Times today that one of my problems if I were to run for president, I like Republicans. OK, well, bless me father for I have sinned," Biden said as he made the sign of the cross while speaking to the United States Conference of Mayors. "From where I come from, I don't know how you get anything done. I don't know how you get anything done unless we start talking to one another again."

Biden's defense of bipartisanship offers a preview of a possible message the former vice president could embrace should he run for president in 2020. Biden often touts his work reaching across the aisle during his 36 years in the Senate as well as his two terms as vice president under President Barack Obama.

But Biden has faced criticism from some Democrats for his willingness to praise Republicans. The New York Times reported Wednesday that some local Democratic officials in Michigan were frustrated and felt betrayed after Biden heaped praise on Republican Rep. Fred Upton of Michigan during a paid speech in his district just weeks before the election without endorsing his Democratic opponent.

"Fred Upton, I went out and spoke at an event and he was there, and I praised him, and he was in the race, but I praised him about the fight against cancer. His vote and his ability to join me," Biden said, referring to the 21st Century Cures Act, which Upton helped craft. "He stepped up. He and three other Republicans stepped up and helped us pass it. And so I acknowledged it, and now I'm, I don't know what I am."

Read More