(CNN) Dr. Ronny Jackson, the embattled pick to lead the Veterans Affairs agency, will continue fighting for his nomination, according to senior administration officials, despite allegations of misconduct and hints from President Donald Trump that he could withdraw himself from contention.

The White House feels Jackson is being "railroaded" by the claims he oversaw a toxic work environment and drank to excess. Instead, the officials insisted Jackson would push back forcefully and said Trump was not wobbling over his selection. Trump met with Jackson on Tuesday in the Oval Office after saying it was "totally (Jackson's) decision" whether to continue the confirmation process.

The White House began publicly defending Jackson after the meeting, distributing praise from former administrations and claiming the raft of allegations came from a disgruntled ex-employee.

"Dr. Jackson's record as a White House physician is impeccable. He has improved unit morale, received glowing reviews and promotions under Republican and Democrat presidents," read a statement from a White House official, who declined to be named.

That confidence aside, Jackson's viability as a nominee appears in question. Multiple Republican and Democratic lawmakers have said the allegations are troubling, and his nomination hearing was postponed indefinitely.

Read More