WASHINGTON -- House Speaker John Boehner (R-Ohio) is throwing a lifesaver to House Majority Whip Steve Scalise (R) as the Louisiana congressman faces a deluge of criticism and questions over a 2002 speech he gave to a white supremacist group.

"More than a decade ago, Representative Scalise made an error in judgment, and he was right to acknowledge it was wrong and inappropriate. Like many of my colleagues on both sides of the aisle, I know Steve to be a man of high integrity and good character. He has my full confidence as our Whip, and he will continue to do great and important work for all Americans," Boehner said in a Tuesday statement.

House Majority Leader Kevin McCarthy (R-Calif.) is also standing by Scalise.

"Congressman Scalise acknowledged he made a mistake and has condemned the views that organization espouses," McCarthy said in a statement released moments after Boehner's. "I've known him as a friend for many years and I know that he does not share the beliefs of that organization."

The show of support from GOP leaders came as Scalise has found himself under fire for being a guest speaker at a 2002 meeting of the European-American Unity and Rights Organization, a group founded by former Ku Klux Klan leader David Duke. A Louisiana political blogger this week unearthed evidence of Scalise's attendance at the event, and from there the news went viral.

Scalise, who was a state lawmaker at the time, maintains that he spoke to any groups who would have him and says he didn't know that EURO was affiliated with racists and neo-Nazi activists. He issued a new statement Tuesday, saying he regretted speaking to the group.

"Twelve years ago, I spoke to many different Louisiana groups as a state representative, trying to build support for legislation that focused on cutting wasteful state spending, eliminating government corruption, and stopping tax hikes. One of the many groups that I spoke to regarding this critical legislation was a group whose views I wholeheartedly condemn. It was a mistake I regret, and I emphatically oppose the divisive racial and religious views groups like these hold. I am very disappointed that anyone would try to infer otherwise for political gain," said Scalise.

He continued, "As a Catholic, these groups hold views that are vehemently opposed to my own personal faith, and I reject that kind of hateful bigotry. Those who know me best know I have always been passionate about helping, serving, and fighting for every family that I represent. And I will continue to do so."

Duke described Scalise as "a pretty nice guy" and "a family man" in a Monday night interview with The Huffington Post. He also said it seemed a bit strange that Scalise -- who had a friendly relationship with Duke's campaign manager Kenny Knight, the EURO event's organizer -- claims he didn't know what the group's message was about.