Washington(CNN) The House Oversight Committee is investigating the White House's handling of Rob Porter's employment, chairman Trey Gowdy told CNN Wednesday.

"We are directing inquiries to people that we think have access to information we don't have. You can call it official. You can call it unofficial. Those words don't mean anything to me. What means something to me is I'm going to direct questions to the FBI that I expect them to answer," Gowdy, R-South Carolina, told CNN "New Day" co-anchor Alisyn Camerota.

He added that he was "troubled by almost every aspect" of Porter's employment at the White House.

"How do you have any job if you have credible allegations of domestic abuse? Again, I am biased toward the victim," Gowdy said.

Porter resigned last week from his role as staff secretary amid domestic abuse allegations, which he denies.

FBI Director Christopher Wray testified Tuesday on Capitol Hill that the FBI briefed the White House multiple times on its investigation into Porter last year, contradicting the White House's accounting of when it learned of the allegations against the former staff secretary and the status of the FBI's background check investigation to determine whether Porter could be granted a permanent security clearance.

CNN reported that some senior aides -- including chief of staff John Kelly and White House counsel Don McGahn -- were aware of both the allegations against Porter before they became public and of his troubles obtaining a security clearance. Two of Porter's ex-wives had detailed the allegations to the FBI during a routine background check into Porter.

"I would want to know from Don McGahn and General Kelly and anyone else: What did you know, from whom did you hear it, to what extent did you hear it and then what actions, if any, did you take? The chronology is not favorable from the White House," Gowdy said.

Gowdy sent a formal letter to Wray later Wednesday requesting further details, including what "derogatory information" was made available to the White House on Porter, when and to whom, by the end of the month.

North Carolina Republican Rep. Mark Walker, a prominent House conservative, praised Gowdy's move.

"Any man who abuses a woman or exhibits abusive behavior should not be in public service," Walker, who also sits on the House Oversight Committee, said in a statement Wednesday.

"Having spent years in ministry and offering counseling to survivors, such abuse must be confronted -- not protected by friends or people in power," Walker continued. "I am proud of my good friend Trey Gowdy as he leads the effort to get answers."