Nearly three-quarters of the political groups subjected to extra tax inspection during a recent IRS scandal were not identifiably opponents of the White House, officials have revealed.

Republicans have called for a special prosecutor to investigate what they call "attacks on American democracy" after it emerged that rightwing groups were singled out for special treatment by the Inland Revenue Service.

But in testimony before Congress, two senior civil servants brought in to clean up after the affair gave evidence that suggests the reality may have been more complex than a simple case of political bias.

The IRS office in Cincinnati which decides whether to exempt such groups from income tax singled out 72 of them for scrutiny because they were openly affiliated with the Tea Party movement, together with 24 others whose names included associated labels such as "patriot".

To qualify for tax exempt status such groups have to show they are not directly backing a political candidate but they are allowed to campaign on general "civic issues".

However a further 226 other political groups were also placed in the same review whose affiliations were not immediately apparent from their name alone, which is often the case among liberal campaign groups. It remains unknown how many of these were in fact Democrat-leaning groups, partly because individual names cannot be publicly released under IRS confidentiality laws.

The IRS did reveal there had been an explosion in groups of all political persuasion seeking to qualify for this type of tax exemption after a relaxation in campaign finance rules meant this would also allow them to keep the identity of their donors secret. In total 3,357 applications were made in 2012 compared with 1,735 in 2010 before the law changed.

Democrats on the committee seized on the new data and said it revealed the hidden story behind the IRS scandal – that political parties were exploiting the tax code to hide their donors.

"There isn't a single person up here [on this committee] who doesn't understand what is going here and what it says about the hidden nature of political donations," said Marcy Kaptur of Ohio. "This is what demands serious attention by the IRS."

Kaptur praised "some smart people in the IRS" for "doing their job" and trying to use overt political affiliations to screen out non-profits that might be fund-raising groups in disguise, although she said there was insufficient clarity in the law about what classed as political activity.

Acting IRS commissioner Danny Werfel told the committee he had launched a comprehensive review to restore trust in the agency because "the use of certain political labels resulted in some applications being inappropriately singled out".

"The agency stands ready to confront the problems that occurred, hold accountable those who acted inappropriately, be open about what happened and permanently fix these problems so that such mis-steps do not occur again," he said. "We need to show the utmost fairness and impartiality … and there was a fundamental failure [to do this] by the IRS management."

But Republican critics saw his testimony and that of treasury inspector general J Russell George as more evidence to support their view that the practice amounted, at best, to inadvertent political bias and may even have been sanctioned by the Democrat administration.

Mario Diaz-Balart of Florida said: "When government targets individuals for their beliefs this is an affront for democracy and an attack our freedoms. It's a basis of our country that you should not fear the government because of your beliefs".

Fellow Democrat Jose Serrano added: "We need to have a have look at why we allow groups involved in political funding to have special status. Freedom of speech does not require this kind of tax advantage."

Delays in granting the tax exempt status are still affecting 132 of the political advocacy groups, some of whom have waited over two years to clear through the IRS system – leading several to go bankrupt.

Republican committee chair, Hal Rogers, compared the affair to a McCarthyite witch-hunt. "Having an enemies list harkens back to a dark page in our past," he said.

Subcommittee chairman, Ander Crenshaw, added: "When an entire office singles out conservative groups and bullies them and harasses them, that's not bad management; that's someone gone wrong. And if no one steps up and says that's wrong, that's a culture of abuse."