Bill O’Reilly, historian, has had quite the week. On Tuesday night, responding to Michelle Obama’s speech at the Democratic National Convention, the Fox News host pointed out—just as a point of interest!—that slaves working on the construction of the White House’ were joined by free black, white, and immigrant labor, and that they were “well-fed and had decent lodgings provided by the government.”

On Wednesday night, O’Reilly brought up the subject again, expressing wonder at the pushback he got for his comments. He berated the media, professing an innocent historical investment in the topic. “As any honest historian knows, in order to keep slaves and free laborers strong, the Washington administration provided meat, bread, and other staples, also decent lodging on the grounds of the new presidential building. That is a fact. Not a justification, not a defense of slavery. Just a fact,” O’Reilly thundered.

Historical rebuttals to O’Reilly have not been scarce. In the Washington Post, Peter Holley rounded up evidence regarding the use of slave labor in the White House’s construction, questioning O’Reilly’s assertion that treatment of such workers was humane. In the Atlantic, David A. Graham pointed out an Abigail Adams letter from 1800 describing the adverse conditions she observed among slaves working in the capital.

Graham also put O’Reilly’s remarks in context, pointing to a long, ugly tradition in which evidence of the supposedly benign nature of slavery has been used to justify the institution. On his blog, Civil War Memory, historian Kevin M. Levin further articulated that argument by including a scanned page from a Virginia history textbook, used in classrooms as late as the 1970s, that describes a “feeling of strong affection” that supposedly “existed between masters and slaves in a majority of Virginia homes.” O’Reilly has either missed a huge strain in American intellectual history or is disingenuously pretending that this history of denial has nothing to do with his comments.

The O’Reilly dispute has strengthened a feeling I’ve had since Jamelle Bouie and I produced our History of American Slavery podcast for Slate in 2015. Namely, that to argue about the conditions particular enslaved people lived under—to point to “happy” or “unhappy” slaves; to swear that on your ancestor’s plantation, enslaved people sang and danced; to make supposedly rational points about how the provision of good nutrition and medical care was in an enslaver’s best economic interest—will always be to miss the point altogether. As we wrote, all of these conversations are distractions from the real issue: “Slavery, as a system, legalized and codified the slaveholder’s control over the enslaved person’s body.”

Harriet Beecher Stowe used plot to make this argument back in 1852, in Uncle Tom’s Cabin. At the beginning of the book, the characters Tom and Eliza are held in bondage by Arthur and Emily Shelby, two Kentucky farmers who are relatively kind to the people they own. But the Shelbys fall into dire financial straits and decide to sell Tom and Eliza’s son Harry to a slave trader. This decision sets the plot into motion; Eliza escapes north with Harry, and Tom is sold.

The Shelbys are “kind,” but because they own people under a system of hereditary chattel slavery that turns humans into property, their kindness means very little in the grand scheme of things. Once Tom is out of their hands, anything could happen to him: The law offers little protection. He lands in another relatively good situation with the St. Clares—and then, with his sale to the sadistic Simon Legree, a terrible one. His fate has been sealed by his status within the system.

On Wednesday, historian Caleb McDaniel tweeted an excellent thread about the O’Reilly remarks, arguing that they “spotlight the pedagogical challenges for teachers of the history of slavery.” Historians are interested in understanding local permutations of American slavery, because “slavery was not monolithic, and living conditions for enslaved people varied across space and time.” But in talking about this type of variance, scholars run the risk of offering fodder for future O’Reillys to use, disconnected from all context. If you want to hear This particular person, living in Virginia in the 18th century, managed to buy himself out of slavery, using the wages a slaveholder allowed him to earn as Slavery wasn’t that bad, that’s what you’ll hear.

Moreover, McDaniel pointed out, scholars have recently been trying to understand ways that enslaved people worked collectively to try to improve their own conditions, using resistance to buy privacy, time, the right to raise their own food, and other allowances. (Stephanie M. H. Camp’s book Closer to Freedom: Enslaved Women and Everyday Resistance in the Plantation South is an example of this kind of scholarship.) But, McDaniel wrote, talking about this aspect of slavery in the classroom is extremely difficult.

6. Yet the reality is that some students will hear talk of any small improvements in enslaved people's lives the way O'Reilly hears it. — Caleb McDaniel (@wcaleb) July 27, 2016

8. Especially easy for a student to hear a white prof's points about enslaved people's small victories & interpolate O'Reilly denialism. — Caleb McDaniel (@wcaleb) July 27, 2016

McDaniel’s suggestion: Always emphasize the nature of the system, which allowed slaveholders to sell parents and children apart from each other, covered up abuse and murder, and allowed daily rapes to occur unpunished.

12. The daily working lives of enslaved people varied; the daily fear and threat of sale away from families and the familiar did not. — Caleb McDaniel (@wcaleb) July 27, 2016

To discuss local conditions in service of understanding the way slavery unfolded is an important way for us to comprehend how the institution worked—so long as you always keep the larger injustice in mind. O’Reilly either can’t understand this or doesn’t want to. I wonder which it is.