When Roger Ailes saw his words in print, the stark accusation that President Obama is lazy, he was momentarily taken aback.

“I looked at it and thought, someone’s made outrageous statements about our leaders, and that someone sounded like me,” the Fox News chairman told me in an interview. He promptly offered an explanation for that slam and for his dismissal of Vice President Biden as “dumb as an ashtray.”

“Anybody who knows me knows that half the time I’m saying things with a somewhat humorous overtone,” Ailes says.

So is he backing away from the incendiary comments? Not a chance.

“Are every one of those statements true? Yes,” Ailes says. “Should I have said them? Well, that’s a debate.”

It’s a debate that has been kick-started by the new Zev Chafets biography, Roger Ailes Off Camera. It was Chafets who reported the assessment of the president, which Ailes says was rooted in Obama’s comment to Barbara Walters that he feels a certain laziness in himself that stems from growing up in Hawaii.

Some African-Americans were appalled. Van Jones, a former Obama White House aide, said on CNN that Ailes’s comment was “disgusting” and “a racial charge that some people find offensive.”

Ailes was anxious to hit back. “I don’t pay a lot of attention to communist infiltrators,” he says. “Van Jones has one job, to stir up racism whether he can find it or not.”

Jones has long since acknowledged saying in the early 1990s, when he was in his 20s, that he was a communist. Ailes acknowledged to me that the CNN contributor has moved away from that self-description.

Asked for comment, Jones says: “Ludicrous name calling from the 1950s playbook doesn’t change the facts. People are tired of hearing ‘lazy Negro’ stereotypes about a president who has worked so hard that his hair has turned white.”

Saying that Obama, who drew flak for his recent golfing weekend, hits the links less than Bill Clinton and Dwight Eisenhower—and far less than John Boehner—Jones asks: “In which book or interview does Ailes call these men lazy?”

Ailes’s decision to cooperate extensively with Chafets stands in contrast to his refusal to talk to New York magazine’s Gabriel Sherman, who is completing what is expected to be a more critical biography. It was something of a preemptive strike.

“I picked Zev,” Ailes says. “He’s an actual author, a good journalist. He’s mature and thoughtful and not out to hurt people. I’m not saying anything about the other guy.”

One of the biggest changes at Fox is Ailes’s decision to drop Sarah Palin as a high-profile contributor after offering her a new contract worth a fraction of the $1 million a year she had been making. But he would not confirm Chafets’s account that Ailes believes Palin has lost her appeal.

“I like Sarah Palin,” Ailes says. “I think she got pounded harder by the media. The only people who got it worse were Richard Nixon and George W. Bush, and that was over a longer period of time.

“Sarah would say she probably was responsible for some of the bad press she got and made some mistakes along the way.”

So why did he cut her loose?

“Nothing’s forever. We purposely took a general cooling-off after the election. Everyone was exhausted from it.”

Fox also axed Dick Morris, who repeatedly predicted that Mitt Romney would beat Obama in an electoral landslide. Ailes says he likes Morris but that “Dick made a lot of mistakes in this thing. He looked like a jerk. He apologized.”

I ventured that Fox, in dropping those commentators while hiring former Massachusetts senator Scott Brown, seemed to be moderating its image. Ailes scoffed at the notion but added, “I also got hit from the right because I hired Dennis Kucinich,” the former Democratic congressman.

Ailes, of course, worked for such GOP presidential candidates as Nixon, Ronald Reagan, and George H.W. Bush during his political consulting days. But he disputes Chafets’s assessment that he is a “Republican” and a “conservative”—more conservative than Fox, in fact—whose views are reflected by the network. Ailes invokes his core mantra, that Fox is fair and balanced, not conservative.

“Do I take alternative points of view? Yes, because all the media are going in another direction,” he says. “My job is to sometimes be the balance.”

I noted that his criticism of Obama and Biden hardly seemed balanced. Without missing a beat, Ailes said some journalists overlook that he also called Newt Gingrich a “prick” in his interviews with Chafets.

So does Ailes consider himself a Republican? He declined to comment but said many television news figures are Democrats and never face that question. In his case, he says, “it’s ‘what about you, man?’”