The Known World has no real plot; rather, it glides back and forth in time, zooming in on the lives of its characters. Set in fictional Manchester County, Virginia, the story begins with the premature death of Henry Townsend. A former slave whose freedom is bought by his father Augustus, Townsend comes to own thirty-three slaves. In one scene, he admits to making his first purchase, and his parents are horrified. (Augustus, a carpenter and walking-stick carver, once used a boxed shipment of his sticks to smuggle a runaway slave to freedom, a vignette that echoes the true story of Henry Box Brown.) When Henry tells Augustus of his purchase, the father replies, "You could not have hurt me more if you had cut off my arms and my legs." He then attacks Henry with one of his sticks, breaking his shoulder.

Moses, Townsend's first slave, is an overseer who eats dirt as a way of tracking the life of the earth he helps to farm. ("This was July, and July dirt tasted even more like sweetened metal than the dirt of June or May.") When Townsend buys him, Moses is bewildered by the fact "that indeed a black man, two shades darker than himself, owned him and any shadow he made" and, reflecting on this strange irony, wonders, "Was God even up there attending to business anymore?" Jones's portraits of Townsend's slaves are full and arresting: mule-kicked Alice, who wanders the night issuing forth a kind of mystical babble; Stamford, whose pursuit of "young stuff" is a comical, shameless, self-destructive, and poignant attempt at surviving a life in bondage; and Elias, whose attempt to flee leaves him without part of an ear.

William Robbins, the largest slave owner in Manchester County, is no simple villain but "one of the few white men who would not suffer from sitting across from a black man." He mentors Henry, loves a black woman, but nevertheless is a strict and cruel enforcer of the slave owner's code. "You are the master," he instructs Henry, "and that is all the law wants to know." Although Henry had "wanted to be a better master than any white man he had ever known," when he dies, his slaves cry no tears.

In the end, Jones's story is as much about Manchester County as it is about Townsend and his slaves. Like Faulkner's Yoknapatawpha County, it is a fictional, even mythical place that is nevertheless imagined down to the tiniest detail. It is "the largest county in Virginia, a place of 2,191 slaves, 142 free Negroes, 939 whites, and 136 Indians, most of them Cherokee but with a sprinkling of Choctaw." While white elites like Robbins compete for power, Manchester's lower classes carve out their places as best they can. The sheriff, John Skiffington, abhors slavery but owns a slave that a spiteful relative gave to him. All the while, an antique map of the Americas hangs on his wall and is labeled "The Known World." Supposedly three hundred years old and wildly inaccurate—"North America on the map was smaller than it was in actuality, and where Florida should have been, there was nothing"—the map serves as an ironic comment on the strange, insular, upside-down world of Manchester County, a place that by the end of the novel has disappeared.