Story highlights The president will appoint a new acting commissioner this week, a source says

President Obama says "Americans have a right to be angry ... and I am angry"

IRS identifies two "rogue" employees, congressional source says

The former acting IRS commissioner will testify before a House committee on Friday

The nation's top tax collector resigned Wednesday amid controversy over Internal Revenue Service targeting of conservative groups that applied for federal tax-exempt status.

The official, acting IRS Commissioner Steven Miller, was aware employees were targeting conservative groups in May 2012, according to the agency.

But Miller, then the agency's deputy commissioner, didn't tell Congress about it when he testified before an oversight committee in July -- despite being questioned on the issue. He was named acting commissioner in November.

An administration official told CNN on Thursday that Obama will appoint a new acting commissioner this week.

Announcing the resignation to reporters Wednesday, President Barack Obama said the agency's misconduct was "inexcusable."

"Americans have a right to be angry about it, and I'm angry about it," Obama said.

"It should not matter what political stripe you're from. The fact of the matter is, the IRS has to operate with absolute integrity," the president said.

JUST WATCHED IRS acting commissioner forced out Replay More Videos ... MUST WATCH IRS acting commissioner forced out 02:51

JUST WATCHED Rep. Issa on IRS: Obama set right tone Replay More Videos ... MUST WATCH Rep. Issa on IRS: Obama set right tone 02:08

JUST WATCHED Gergen: IRS scandal not a partisan issue Replay More Videos ... MUST WATCH Gergen: IRS scandal not a partisan issue 03:04

JUST WATCHED FBI to investigate IRS conduct Replay More Videos ... MUST WATCH FBI to investigate IRS conduct 03:52

Obama pledged to work "hand in hand" with Congress as it investigates the matter, and vowed new safeguards will be put in place at the IRS so that "this doesn't happen again."

Despite his resignation, Miller is expected to testify Friday at a hearing before the House Ways and Means Committee, according to a source familiar with the proceedings.

In an internal message to IRS employees obtained by CNN, Miller said he would be stepping down as commissioner in early June.

"This has been an incredibly difficult time for the IRS given the events of the past few days, and there is a strong and immediate need to restore public trust in the nation's tax agency," Miller wrote.

'Rogue' employees

News of Miller's resignation followed revelations that the IRS has identified two "rogue" employees in the agency's Cincinnati office as being principally responsible for the "overly aggressive" handling of requests by conservative groups for tax-exempt status, a congressional source told CNN.

Miller said the staffers have already been disciplined, according to another source familiar with Miller's discussions with congressional investigators. The second source said Miller emphasized that the problem with IRS handling of tax-exempt status for tea party groups was not limited to these two employees.

Miller met with Senate Finance Committee Chairman Max Baucus of Montana on Tuesday to discuss an appearance before Congress.

Asked Wednesday in a Senate hallway about his meeting with Miller, Baucus told CNN, "I did not learn as much from the meeting as I would have liked."

"I told him that it was in his best interest to be totally cooperative -- that it's often the cover-up that causes more problems than the original malfeasance," the senator said. "And just to be totally straight with me and everybody, and he said he would."

Meanwhile, Republican congressional leaders on Wednesday accused Obama's administration of potentially criminal behavior in the handling of requests for tax-exempt status from conservative groups.

Senate Republican Leader Mitch McConnell suggested criminal behavior had occurred, saying that the "very serious" allegations involve "an effort to bring the power of the federal government to bear on those the administration disagreed with in the middle of a heated national election."

"It actually could be, could be criminal and we are determined to get the answers," McConnell said.

House Speaker John Boehner was more definitive.

"My question is, who's going to jail over this scandal?" he asked.

Boehner told reporters that "clearly someone violated the law" in what an IRS inspector general's report described as delayed processing of applications by groups associated with the political right wing.

Multiple investigations

U.S. Attorney General Eric Holder, who ordered a criminal investigation into the situation, said Wednesday at a congressional hearing that investigators will look at the conduct of IRS offices nationwide.

"The facts will take us where ever they take us," he said.

Holder said the investigation will also examine whether IRS officials lied to Congress about the singling out of conservative groups.

While the allegations originated in the Cincinnati office, the Justice Department inquiry is based out of Washington, Holder said.

The comments came as all 45 Senate Republicans sent the White House a letter that called for the administration to "comply with all requests related to congressional inquiries without any delay" involving the controversy.

The letter called the scandal "yet another completely inexcusable attempt to chill the speech of political opponents and those who would question their government, consistent with a broader pattern of intimidation by arms of your administration to silence political dissent."

The clearly coordinated attacks were part of a GOP effort to increase pressure on the Obama administration over the controversy, one of three potential scandals that has the White House on the defensive less than four months into the president's second term.

According to the report by the agency's inspector general released Tuesday , the IRS developed and followed a faulty policy to determine whether the applicants were engaged in political activities, which would disqualify the groups from receiving tax-exempt status.

The controversial move began in early 2010 and continued for more than 18 months, the report said, declaring that "the IRS used inappropriate criteria that identified for review Tea Party and other organizations applying for tax-exempt status based upon their names or policy positions instead of indications of potential political campaign intervention."

IRS officials, according to the report, did not consult anyone beyond the agency about the development of the additional screening criteria. They believed that the criteria they came up with were a screening shortcut meant to help with the influx of applications, the report said.

The controversial actions began after the 2010 Supreme Court ruling in the Citizens United case that greatly expanded the ability of corporations, unions and other organizations to participate in election spending, though not through direct contributions to candidates or parties.

After the ruling, the number of politically oriented groups seeking tax exempt status as social welfare organizations under section 501 (c) (4) of the federal tax code increased greatly at a time when the federal government, including the IRS, was dealing with austerity measures that reduced or stagnated personnel and resources.

The IRS' top watchdog found that the criteria used to flag potential political applications resulted in substantial delays and the request of unnecessary information from the groups.

'Be On the Look Out'

Among the criteria used by IRS officials to flag applications was a "Be On the Look Out" list, which was discontinued in 2012, according to the report.

The criteria included:

-- Whether "Tea Party," "Patriots" or "9/12 Project" was referenced in the case file.

-- Whether the issues outlined in the application included government spending, government debt or taxes.

-- Whether there was advocating or lobbying to "make America a better place to live."

-- Whether a statement in the case file criticized how the country is being run.

-- Whether it advocated education about the Constitution and the Bill of Rights.

The investigation by the Treasury inspector general for tax administration was initiated after congressional complaints began to surface in the media in 2012 that the IRS was targeting conservative groups and holding up applications.

"Whether the inappropriate criterion was shorthand for all potential political cases or not, developing and using criteria that focuses on organization names and policy positions instead of the activities permitted under the Treasury regulations does not promote public confidence that tax-exempt laws are being adhered to impartially," the report said.

The IRS welcomed the report, saying that it agreed that aspects of its original approach in handling the influx of tax-exempt applications was inappropriate.

"The IRS is required by law to determine if organizations are engaging in a legally permissible level of political activity. Centralizing these cases was necessary to achieve consistent treatment," it said in a statement.

Inefficient management

In a written response included in the report, the IRS commissioner of the Tax Exempt and Government Entities Division said there was no criminal behavior behind the actions of the agents, but rather inefficient management.

"We believe the front-line career employees that made the decisions acted out of a desire for efficiency and not out of any political and partisan viewpoint," the commissioner wrote.

The report's findings indicate that of the 298 cases reviewed by the IRS inspector general as potential political cases not eligible for tax exempt status: 72 contained the name "tea party," 11 contained "9/12" and 13 contained the word "patriots," according to the report. There were 202 cases that did not contain any such reference.

Of those applications still open for review, 160 cases were open from 206 days to more than three years -- through two election cycles.

Among the recommendations made by the Treasury inspector general: The IRS must better document reasons why applications are chosen for review, develop a process to track requests for assistance, develop and provide training to employees before each election cycle and immediately resolve outstanding cases.

The report also called on Treasury to develop guidelines to explain social welfare activity -- the primary factor in obtaining tax-exempt status.