Washington (CNN) Michael Steele shot back Saturday at a Conservative Political Action Conference official for a racial comment he made about the former Republican National Committee chairman at a dinner Friday night.

At the Ronald Reagan dinner, CPAC communications director Ian Walters said Steele was elected RNC chair "because he's a black guy."

"We were somewhat lost as a group, we had just elected the first African-American president, and that was a big deal and that was a hill that we got over and it was something that we were all proud of and we weren't sure what to do, and in a little bit of cynicism what did we do? This is a terrible thing. We elected Mike Steele to be the RNC chair because he's a black guy, that was the wrong thing to do," Walters said.

Steele told MSNBC on Friday that Walters called him and tried to apologize, but he rejected the apology.

"He did call and he tried to explain himself and said what he was thinking, he relayed it back to Barack Obama's election, and then he said at one point, 'I apologize,' and I just said, you know, that's just not acceptable, that's not enough," Steele recalled.

Read More