House Speaker John Boehner, R-Ohio, said the House has no plans to impeach President Barack Obama, but that doesn’t mean the idea isn’t lurking in the wings. Yes, House Republicans have voted to sue the president on the grounds that he singlehandedly rewrote his signature health care law, but with conservatives now calling him "lawless" and "out of control," a lawsuit might not satisfy.

Fox News analyst Juan Williams accused Republicans of demonizing Obama in an Aug. 3, 2014, broadcast of Fox News Sunday . Host Chris Wallace pressed Williams on whether he thought Republican opposition to the president was racial.

"All I can do is look at the numbers," Williams said. Then, in a heated exchange with another panelist, Michael Needham of the conservative group Heritage Action, Williams said, "The core constituency, the people who want him impeached, they’re all white, and they’re all older, and guess what, they’re all in the far right wing of the Republican party."

We decided to find the numbers and see if they matched Williams’ claim that the desire for impeachment rests with white, older, Republican voters.

The polls

We found three polls that asked about impeachment. One from CNN/ORC reported that 33 percent of the public at large thought Obama should be impeached (with a margin of error of 3 percent). Among conservatives and Republicans, the share jumped to about 56 percent.

The CNN/ORC poll found no significant variation among the age groups. The numbers hovered around 33 percent.

There was a sharp racial divide. About 40 percent of whites said they favored impeachment, while just 17 percent of non-whites took that view. (The margin of error was higher for non-whites -- plus or minus 6.5 percentage points.)

Just 13 percent of Democrats supported impeachment.

Fox News did its own survey . Fox put overall support for impeachment at 36 percent. The age factor showed a slightly greater skew than in the CNN poll, with just 28 percent in favor among people under 35 and older voters coming in closer to 40 percent.

Fox also found that about 40 percent of whites liked the idea of impeachment. They reported that 20 percent of African-Americans support impeachment. (The margin of error on was plus or minus nine percentage points.)

Among people who described themselves as "tea partiers," support for impeachment jumped to 68 percent.

The Huffington Post partnered with YouGov to conduct an online survey and like the other polls, reported that about a third of the public favored impeachment. Support was strongest among Republicans, at 68 percent, and lowest among Democrats at 8 percent (with no reported margin of error). As with the Fox poll, this one found more interest in impeachment among older people; 47 percent of those 65 and up liked the idea.

In terms of race, 40 percent of whites, 10 percent of African-Americans and 26 percent of Hispanics leaned in favor, but the number of responses among non-whites was very low and no one should put much faith in those exact figures.

What the experts say

The general sense from the political scientists we reached is that Williams had a point but went overboard. The main problem had to do with the word "all."

"That's a categorical assertion, one that I cannot support," said Christopher Parker, a political scientist at the University of Washington. "However, I will say that this group of people is more likely than any other group of folks to wish to see the president impeached."

Theda Skocpol, a professor of government and sociology at Harvard University, also noted that Williams had taken "a few everyday language shortcuts, but he is basically right."

"In our research in 2010-11, we found that conservative Republicans/tea party sympathizers are overwhelmingly white and definitely tend to be older," Skocpol said. "Now, keep in mind that all Republicans tend, statistically speaking, to be older and whiter than other Americans. So any time you talk about Republicans, you are talking older and whiter. And of course, only Republicans tell pollsters in a majority that they favor impeachment."

William Miller, a political science instructor at Flagler College in Florida who has written about the tea party and the Republican Party, generally agreed with Skocpol. Both the tea party and people supporting impeachment tend to be older and whiter, but "there are pockets of support in other groups," he said.

Our ruling

Williams said that the people who want to impeach Obama are all white and all older and all are in the right wing of the Republican Party. "All" is going too far.

But the publicly available polling shows that the people who support impeachment are more likely to be white, conservative and older. In one poll, about a third of the public supported impeachment, while about two-thirds of tea partiers said they did. In another, 40 percent of whites supported impeachment compared to 10 percent of African-Americans.

The polling data bears out the general thrust of Williams' claim. We rate the statement Mostly True.