House Oversight Committee Chairman Trey Gowdy surprised the political world this week by announcing Wednesday that he wouldn't seek re-election later this year.

'My wife hates it when I say this, but I was a pretty good prosecutor, I think. But I've been a pretty lousy politician,' Gowdy said Sunday on Face the Nation.

Gowdy, a South Carolina Republican first elected in 2010, became a national figure as the chairman of the House Benghazi Committee, which probed the deaths of four Americans at a diplomatic compound in 2012, while Hillary Clinton was serving as secretary of state.

Scroll down for video

House Oversight Committee Chairman Trey Gowdy joined the growing ranks of Republican lawmakers who say they won't run again later this year

Rep. Trey Gowdy (left) told CBS News' Margaret Brennan (right) that he was a 'lousy politician' because he could see 'multiple sides of a single issue' and wasn't concerned with always 'winning'

Despite that being branded as a partisan witch hunt by Democrats – and House Majority Leader Kevin McCarthy infamously remarking that Clinton looked 'unbeatable' until the Benghazi committee brought down her poll numbers – Gowdy indicated he didn't have the appetite for the hyper-partisanship that exists on Capitol Hill these days.

'I just, I see multiple sides of a single issue,' Gowdy explained.

Gowdy, a former U.S. attorney, said he plans to go back into the justice system upon retirement from Congress.

The Republican pointed out that he has many friends across the aisle.

'And the fact that someone disagrees with me, does not make me challenge their love of the country. It doesn't make me believe that they're corrupt ... I don't think the end justifies the means.'

'I think the manner in which we get places matters, and in politics too often winning is the only thing that matters,' Gowdy continued. 'And look, every hero I have has lost. Every one of the them. So losing is not the worst thing in the world.'

'Not knowing what you believe and not caring enough about it to fight for it – that's the worst thing in the world,' he added.

CBS' Margaret Brennan followed that up by asking the South Carolina Republican if he believed he had served justice during his time in the House.

'Not like I did in my previous job,' said Gowdy. 'I tried.'

'There's a reason we throw out search warrants even though we find the murder weapon. There's a reason we throw out confessions even though we think the person did it. The process matter,' Gowdy explained. 'And in politics, it's just about winning. And I can't, I don't want to live like that.'