Amid his Wednesday night rampage at Emanuel African Methodist Episcopal Church in Charleston, South Carolina—killing nine people—21-year-old Dylann Storm Roof reportedly told churchgoers, “You rape our women, and you’re taking over our country, and you have to go.”

Of the assertions in that statement, it’s the first that has a long, deadly history. In the late 19th century, rape was a frequent justification for racist violence. “To palliate this record … and excuse some of the most heinous crimes that ever stained the history of a country,” wrote journalist Ida B. Wells-Barnett on lynchings in her pamphlet Southern Horrors, “the South is shielding itself behind the plausible screen of defending the honor of its women.” Indeed, Wells points to a host of Southern newspapers that defended “lynch’s law” with reference to an alleged epidemic of black-on-white rape. In one editorial, published by the Memphis Daily Commercial, editors declared, “The commission of this crime grows more frequent every year,” and, “There is no longer a restraint upon the brute passion of the Negro.”

As Wells-Barnett would show, however, there was no substance to the charge. “The world knows that the crime of rape was unknown during four years of civil war, when the white women of the South were at the mercy of the race which is all at once charged with being a bestial one,” she writes. In reality, these accusations of rape were often covers for consensual—and taboo—relationships between black men and white women. “Whites could not countenance the idea of a white woman desiring sex with a Negro, thus any physical relationship between a white woman and a black man had, by definition, to be an unwanted assault,” writes historian Philip Dray, describing Wells-Barnett’s argument in his book At the Hands of Persons Unknown: The Lynching of Black America. In one instance, found Wells-Barnett, a black man in Indianola, Mississippi, was lynched for raping the local sheriff’s 7-year-old daughter. When Wells-Barnett went to investigate, however, she found something very different:

Wells traveled to Indianola and met the alleged rape victim, who was no girl but a grown woman in her late teens. The “brute,” Wells learned, had worked on the sheriff’s farm for a number of years and was acquainted with every member of the family. The woman had been found in her lover’s cabin by her father, who led a lynch mob in order to save his daughter’s reputation.

Make any list of anti-black terrorism in the United States, and you’ll also have a list of attacks justified by the specter of black rape. The Tulsa race riot of 1921—when white Oklahomans burned and bombed a prosperous black section of the city—began after a black teenager was accused of attacking, and perhaps raping, a white girl in an elevator. The Rosewood massacre of 1923, in Florida, was also sparked by an accusation of rape. And most famously, 14-year-old Emmett Till was murdered after allegedly making sexual advances on a local white woman.

It’s worth a look at the second part of Dylann Roof’s claim, which informs the first: “You’re taking over our country.” Behind the myth of black rapists was an elemental fear of black autonomy, often expressed by white Southern leaders who unhesitatingly connected black political and economic power to sexual liaison with whites. “We of the South have never recognized the right of the Negro to govern white men, and we never will,” said Sen. Benjamin Tillman on the Senate floor in 1900. “We have never believed him to be equal to the white man, and we will not submit to his gratifying his lust on our wives and daughters without lynching him.”

This fear took its most violent form in the years after the Civil War—when blacks won their freedom—but it’s been with us for centuries. Throughout the antebellum period, whites lived in terror of a challenge to the racial order and believed that black freedom would lead to a world of “Negro domination” where they lived as slaves, or worse. Likewise, during the 1864 election, Northern Democrats attacked Abraham Lincoln as a “black Republican” who sought to debase the white race with miscegenation. Similar accusations were used during the civil rights movement, and in the North, urban whites justified residential segregation—in patterns that still exist—as necessary protection from the violent, sexually aggressive black criminal.

Fetid swamps of Internet racism aside, racist rape accusations have all but faded from American life. But the more elemental fear—of a topsy-turvy world of black dominion—endures in diminished form. You hear it, for instance, in claims that Barack Obama is here to bring “reparations,” to punish a “Colonialist” America, or to build a world where “white kids now get beat up with the black kids cheering.”

It’s tempting to treat Dylann Storm Roof as a Southern problem, the violent collision of neo-Confederate ideology and a permissive gun culture. The truth, however, is that his fear—of black power and of black sexuality—belongs to America as much as it does the South.

Read more of Slate’s coverage of the Charleston shooting.