One Democratic senator isn't going as far as Sen. Elizabeth Warren, who called for the ouster of Wells Fargo CEO Tim Sloan on Tuesday.

Sloan faced a grilling on Capitol Hill about the bank's fake accounts scandal.

"Tim Sloan's trying to do the right thing," Sen. Jon Tester, D-Mont., said in an interview with CNBC's "Power Lunch."

That said, Tester said he's still fighting to make sure that wrongs are righted.

"They screwed up and they need to fix it. They need to fix this culture that's gone on in their banking system for far too long," he said.