Attorney General Eric Holder said Wednesday that the Justice Department investigation into IRS employees singling out conservative nonprofits for extra scrutiny will be a broad, nationwide one based in Washington.

"This is something that we will base in Washington, and that way we can have a better impact nationwide," Holder said in response to tough questioning from lawmakers at a House Judiciary Committee hearing Wednesday afternoon.

Holder said he launched an investigation last Friday into why the IRS subjected conservative groups to more review when they applied for tax-exempt status. The IRS inspector general's report said that a group of low-level staffers in an Ohio office were responsible, and a top IRS official has apologized on their behalf.

Rep. Bobby Scott, D-Va., asked Holder at the hearing whether an "apology" from the IRS protected them from criminal prosecution. Holder answered, "No."

The DOJ investigation will go beyond Ohio and look into any allegations of targeting elsewhere, Holder said. He noted it's possible civil rights laws have been violated. "We will take a dispassionate view of this," Holder said. "This will not be about parties ... anyone who has broken the law will be held accountable."

At the hearing, Holder faced pointed questions from both sides of the aisle over the twin scandals that have dogged the Obama administration this week: the IRS revelations and the seizure of phone records of Associated Press reporters and editors. In his prepared statement at the hearing, Holder mentioned neither topic, instead focusing on the Justice Department's commitment to civil rights, immigration reform and the reversal of sequester cuts.

During the hearing, Holder also said he realized there's been "criticism" of the department's decision to subpoena records for the private and work phones of more than 20 AP reporters and editors without notifying them first. But he added that he was unable to say why the investigation's scope was so large or why it was kept secret from the AP because he had recused himself from the matter along with the rest of the national security division. Deputy Attorney General James Cole signed the subpoena, he said.

Story continues

"I am not involved in the case," Holder said.

Holder said he didn't believe there is any documentation of his recusal, and acknowledged that it would be better practice to document all of his recusals in writing.

Rep. Jim Sensenbrenner, R-Wis., said he was troubled that Holder did not take responsibility for the decision. "We don't know where the buck stops," he said.

On Tuesday, Holder said at a news conference that the national security leak that prompted the department to seize AP phone records was among the most serious he had ever seen.

“I have to say this is ... among the top two or three most serious leaks I have ever seen. It put the American people at risk. That is not hyperbole,” he said. The leak led to an AP story last year about the government foiling a Yemeni-based terror plot to bomb American airliners.

Holder's appearance at the Republican-led House Judiciary Committee hearing was scheduled before the IRS and AP news broke, and was to focus on oversight of the DOJ. On Wednesday he was also questioned about whether there were any intelligence failures in the lead-up to the Boston Marathon bombings, as well as on marijuana laws and hate crime prosecutions.

Holder has long faced criticism from Republicans, some of whom called for his resignation in 2011 over the failed gun-walking Fast and Furious operation on the U.S.-Mexico border. The Justice Department's inspector general cleared him of wrongdoing in that scandal, blaming the botched operation on Arizona federal prosecutors and ATF agents.

Watch Holder's testimony above.