Washington (CNN) The House Oversight Committee is investigating the White House's handling of Rob Porter's employment, chairman Trey Gowdy told CNN Wednesday.

"We are directing inquiries to people that we think have access to information we don't have. You can call it official. You can call it unofficial. Those words don't mean anything to me. What means something to me is I'm going to direct questions to the FBI that I expect them to answer," Gowdy, R-South Carolina, told CNN "New Day" co-anchor Alisyn Camerota.

He added that he was "troubled by almost every aspect" of Porter's employment at the White House.

"How do you have any job if you have credible allegations of domestic abuse? Again, I am biased toward the victim," Gowdy said.

Porter resigned last week from his role as staff secretary amid domestic abuse allegations, which he denies.

Read More