Democrats challenged former Vice President Joe Biden during CNN’s Democrat presidential debate on Wednesday, but in the process, they criticized former President Obama.

Democrats like New York City Mayor Bill de Blasio criticized Democrats for continuing to support the concept of private insurance.

“There’s this mythology that somehow all of these folks are in love with their insurance in America,” he said.

Obama’s former secretary for the Department of Housing and Urban Development (HUD) Julian Castro criticized Obama’s former Department of Homeland Secretary, saying that his concern about decriminalizing crossing the border illegally was merely a “right-wing talking point.”

He also criticized the Obama administration for high rates of deportation of illegal immigrants, challenging Joe Biden for failing to intervene.

De Blasio also pressed Biden on why he did not advise Obama to slow deportation during the administration. “Did you say those deportations were a good idea or did you go to the president and say, ‘this is a mistake, we shouldn’t do it,'” he said. “Which one?”

Biden defended Obama for trying to pass comprehensive immigration reform but vowed that he would “absolutely not” resume similar deportation rates as the Obama administration.

Castro also joined the criticism of the Obama administration, suggesting that the former president had made mistakes in office.

“First of all, Mr. Vice President, it looks like one of us has learned the lessons of the past and one of us hasn’t,” he said to Biden.

Biden reminded Castro that he also served in the Obama administration.

“I never heard him talk about any of this when he was the secretary,” he said pointedly.

But even Biden found himself implicitly criticizing Obama’s proposed Trans-Pacific Partnership (TPP) deal, which Trump killed, when asked if he would rejoin the proposed trade agreement with Asian nations.

“I would not rejoin the TPP as it was initially put forward,” he said.

Biden also criticized the surge of troops in Afghanistan, which took place in 2009 under President Obama.

“I opposed the surge in Afghanistan,” he said, adding that he felt going into Afghanistan was a mistake.

Congresswoman Tulsi Gabbard also criticized both parties for allowing the war in Afghanistan to continue.

“For too long, we’ve had leaders who have been arbitrating foreign policy from ivory towers in Washington without any idea about the cost and the consequence, the toll that it takes on our service members, on their families,” she said.

Ultimately, Biden reminded his political rivals that Obama was the one that chose him as the vice president.

“Everybody is talking about how terrible I am on these issues. Barack Obama knew exactly who I was,” he said, noting that he was fully vetted as vice president. “He chose me, and he said it was the best decision he made. I’ll take his judgment.”

After the debate, Obama’s former Attorney General Eric Holder cautioned Democrats for their attacks against the Obama administration.

“To my fellow Democrats. Be wary of attacking the Obama record,” he wrote on Twitter. “Build on it. Expand it. But there is little to be gained – for you or the party – by attacking a very successful and still popular Democratic President.”