Former U.S. Attorney General Michael Mukasey on Sunday defended President Barack Obama's constitutional authority to exchange five Taliban prisoners of war for U.S. Army Sergeant Bowe Bergdahl, but suggested that Obama should be impeached anyway.

"He broke the law, but I believe that the law itself is unconstitutional," Mukasey said, referring to Obama's apparent failure to give Congress 30 days' notice before approving the prisoner exchange. "Article II [of the U.S. Constitution] makes him commander in chief of the armed forces. These people were in the custody of the armed forces."

While many Republicans have decried Obama's actions as illegal, saying he violated the 30-day notification statute, Mukasey said that the statute itself is not legitimate.

"It's unconstitutional, and Obama said so at the time that he signed it," Mukasey said.

Nevertheless, Mukasey suggested that Congress could still impeach the president for agreeing to release five Taliban prisoners of war in exchange for Bergdahl, saying that elected officials do not need to commit crimes to be impeached.

"Whether you impeach somebody doesn't depend on whether they violate the law," Mukasey said. "The president can stay within his lawful powers and still commit an impeachable offense. He can pardon anybody he wants. If he decided tomorrow to pardon everybody in the U.S. prison system, that would be lawful, but it would raise serious questions about whether he should continue in office. The same is true of the wholesale release of dangerous people."

The U.S. Constitution states that presidents can only be impeached for "treason, bribery or other high crimes and misdemeanors." Under traditional rules of war, the United States government would be required to release all five of the Taliban members in U.S. captivity in Guantanamo Bay when the war in Afghanistan ends. Obama announced in late May that the U.S. is drawing down forces in Afghanistan and preparing to exit the conflict.

During an appearance on ABC's "This Week" on Sunday, House Intelligence Committee Chairman Mike Rogers (R-Mich.) pushed back against calls for an Obama impeachment. When asked by host George Stephanopoulos whether Congress should consider impeachment over the Bergdahl flap, Rogers said "that's a long way down this road," adding it was not appropriate to "start hanging scalps on a pole."