Attorney General Eric Holder is resigning from the Obama administration, capping a tumultuous six-year term marked by several high-profile clashes with Congress.

President Obama formally announced the decision, made public earlier in the day, at the White House late Thursday afternoon. Calling Holder's resignation "bittersweet," Obama touted Holder's record on civil rights, as well as terror and corruption prosecutions.

"Eric has done a superb job," Obama said.

Holder is one of the few Cabinet officials to have stayed on since the beginning of the Obama presidency, and Obama said Holder has agreed to stay on until a successor is confirmed.

Holder, choking up several times during brief remarks at the White House, said: "I will leave the Department of Justice, but ... I will never leave the work."

More On This...

The attorney general, who previously worked in private practice and before that was a federal prosecutor, was one of the most controversial members of the administration. Republicans made clear Thursday they would not miss him in the Obama Cabinet.

"Eric Holder is the most divisive U.S. attorney general in modern history," House Oversight and Government Reform Committee Chairman Darrell Issa, R-Calif., said in a statement. "By needlessly injecting politics into law enforcement, Attorney General Holder's legacy has eroded more confidence in our legal system than any attorney general before him."

Holder was held in contempt of Congress by the Republican-led House in 2012 -- a vote in which Issa played a central role -- for failing to provide key information about the botched Operation Fast and Furious program. He is the only sitting Cabinet member to have been held in contempt of Congress.

Holder and GOP lawmakers routinely clashed, over Fast and Furious but also over his department's handling of the IRS targeting scandal, its civilian prosecution of terror cases, surveillance of media outlets and other issues. His hearing appearances were at-times combative; some lawmakers even sought to impeach him.

During the contempt fight, Sen. John Cornyn, R-Texas, was quoted saying Holder's "arrogance knows no bounds."

In that debate, Holder was blasted by Republicans for not turning over documents related to the gun-running probe along the Southwest border. Democrats decried the contempt push, in an election year, as political.

Holder is the fourth-longest serving attorney general in U.S. history, and the first black attorney general. Rumors about his departure have circulated for a while. According to a Justice official, Holder had discussed his plans with Obama on "multiple occasions" in recent months.

"[He] finalized those plans in an hour-long conversation with the president at the White House residence over Labor Day weekend," the official said. The official added that Holder has "no immediate plans" after leaving the post.

A White House official said the president has not made any final decision about a possible replacement for Holder.

Of the outgoing attorney general, the official said: "Holder’s accomplishments have established a historic legacy of civil rights enforcement and restoring fairness to the criminal justice system. Holder revitalized the Department’s praised Civil Rights Division, protected the rights of the LGBT community, successfully prosecuted terrorists, and fought tirelessly for voting rights, to name a few."

The 63-year-old former judge and prosecutor, who took office in early 2009, grappled with divisive questions on the handling of captured terrorism suspects.

In his first few years on the job, Holder weathered a firestorm over an ultimately-abandoned plan to try terrorism suspects in New York City. The attorney general gave up the effort, but he continued to maintain that civilian courts were the most appropriate venue. He argued that his original plan was vindicated by the successful prosecution in New York of Usama bin Laden's son-in-law -- this week, he was sentenced to life in prison.

Under Holder's watch, the Justice Department also cracked down on news media reporting on national security matters. The department secretly subpoenaed phone records from Associated Press reporters and editors and used a search warrant to obtain some emails of Fox News' James Rosen as part of a separate leak investigation.

Holder eventually agreed to overhaul department policies with regard to dealing with media.

The attorney general was often outspoken in his public remarks. He even referred to America in 2010 as a "nation of cowards" in discussing matters of race. He later lamented that "systemic and unwarranted racial disparities remain disturbingly common."

He was known to weigh in on matters of racial fairness, taking steps to improve police relations with minorities, enforce civil rights laws and remove racial disparities in sentencing. Most recently he became the Obama administration's face in the federal response to the police shooting last month of an unarmed black 18-year-old in Ferguson, Mo. In the shooting's aftermath, he enlisted a team of criminal justice researchers to study racial bias in law enforcement.

A former deputy attorney general in the Clinton administration, Holder was pulled away from private practice to reshape a Justice Department that had been tarnished by a scandal involving fired U.S. attorneys and that had authorized harsh interrogation methods for terrorism suspects.

Fox News' Lucas Tomlinson and Ed Henry and The Associated Press contributed to this report.