The Justice Department doesn’t always bring the hammer down on financial institutions accused of wrongdoing because “innocent people” can suffer from the prosecution attempts, Attorney General Eric Holder said on Thursday.

Responding to a question posed by Oregon Senator Jeff Merkley, Holder no corporation is “too big, no person is too important to be held accountable in a criminal sense — if that is appropriate.”

Holder went on to list over a dozen examples by which the Justice Department secured guilty pleas from corporate executives for a myriad of wrongdoing. But in nearly all of those cases, no one was criminally held accountable for their actions.

Instead, Holder said the department made a “creative” and “aggressive use of civil law” in securing deals by which financial institutions paid seemingly-hefty fines in lieu of criminal prosecution.

Last November, JP Morgan Chase agreed to pay a $13 billion civil fine to the Justice Department for improperly grouping subprime loans in mortgage bond packages that were sold to investors. Although the settlement amounts to half of the bank’s annual profit , Holder marketed the deal as proof that justice had been served.

In some cases, though, banks and Wall Street firms get off scot-free. In 2012, the Justice Department announced it would not criminally prosecute Goldman Sachs executives for misleading investors and Congress about a loan product that contributed to the financial crisis years ago.

Several months later, Holder said that the sheer size of large financial institutions made it difficult for the Justice Department to prosecute wrongdoing because “if you do bring a criminal charge it will have a negative impact on the national economy, perhaps even the world economy.”

Holder stood by his remark on Thursday, although he said it had been “misunderstood.” He clarified by saying that when the Justice Department considers prosecuting a firm for wrongdoing, they first weigh any possible outcome such prosecution may have on “innocent people.”

“If you go after an organization and you put that organization out of business as a result of an indictment…there are innocent people who then get punished, potentially employees and shareholders,” Holder said.

Merkley responded by noting that Americans had grown seemingly dissatisfied with the apparent imbalance of justice with regard to crimes committed by large banks compared to those committed by ordinary people.

Merkley noted that the same week the Justice Department announced a $1.9 billion settlement with the British bank HSBC for laundering money, a news report detailed a prison sentence for a woman whose boyfriend was caught stashing drug money in her attic (Merkley may have been referring to the 1993 case of Stephanie George, whose case was highlighted in a Huffington Post article around the same time the HSBC settlement was announced).

“One involved a few dollars; the other involved hundreds of billions of dollars,” Merkley said. “It just seemed like the sort of thing that sticks in people’s minds as to whether the justice system is not weighted heavily in favor of the powerful.”