"There was no way they could properly value the assets without wiping out the capital in the banking system," Hovde told "Squawk Box." "There was no way to properly manage the assets (and) there was no way to dispose of the assets, without cratering the real-estate markets any further."

(Watch the accompanying video for the full interview...)

(The Treasury is a) "bunch of Goldman Sachs guys," he said. "We have a 35-year-old technology investment banker running the TARP that has no background in financials or in real estate or was around during the last banking crisis."

"Then they switch and change to what should have been done and that's injecting capital into the banks," Hovde said. "However, they've even messed that up."

It was the right decision to start to recapitalize banks, but the fact that the money was given away without more punitive terms—for example allowing some banks to sill pay common stock dividends—has just created more problems, he said.

"Now they've created this aura where if you don't take the capital you're being deemed as an institution that couldn't get it so you're in trouble. It's insanity," he added.

Banks are asking "why not" take the money at terms that are more favorable than getting capital from an investors like Warren Buffett, he said.