With every step into the overgrown thicket, Mae Miller’s breathing becomes more labored. “My heart is beating so fast,” she says. “I can’t believe I’m back here.” It’s not the unsteady footing in this field in Gillsburg, Miss., that’s giving her pause; it’s the memories. Some 50 years ago, Miller says she and her parents, Cain and Lela Wall, and her six siblings were held like slaves on this land and surrounding farms. “We been though pure-D hell,” she says today. “I mean hell right here on earth.”

The story that Miller, 63, and her relatives tell is a sepia-toned nightmare straight out of the Old South. For years, she says, the family was forced to pick cotton, clean house and milk cows—all without being paid—under threat of whippings, rape and even death. They say they were passed from white family to white family, their condition never improving, until finally, hope that life would ever get better was nearly lost. Technically, the Walls were victims of “peonage,” an illegal practice that flourished in the rural South after slavery was abolished in 1865 and lasted, in isolated cases like theirs, until as recently as the 1960s. Under peonage, blacks were forced to work off debts, real or imagined, with free labor under the same types of violent coercion as slavery. In contrast with the more common arrangement known as sharecropping, peons weren’t paid and couldn’t move from the land without permission. “White people had the power to hold blacks down, and they weren’t afraid to use it—and they were brutal,” says Pete Daniel, a historian at the Smithsonian Institution and an expert on peonage.

Get push notifications with news, features and more.

By the 1940s, according to records in the National Archives, only rare cases of long-term peonage survived, mostly in rural areas and small towns. That places the Wall family—who say they lived in drafty shacks with grass-filled pallets for beds on white-owned farms until 1961—among a tiny minority. The family’s story might not be known at all if it weren’t for the work of a New Jersey lawyer, Deadria Farmer-Paellmann. In 2001 she began a national effort to claim reparations from corporations that long ago profited from slavery. She scoured the country for descendants of slaves and learned about the Wall family from Louisiana genealogist Antoinette Harrell. Farmer-Paellmann still marvels that the end of slavery had made no practical difference in their lives, even after the advent of TV and jet travel. “They didn’t know blacks were free, that’s what’s so incredible about their story,” says Farmer-Paellmann. “They thought freedom was for whites only.”

Mostly out of fear, but also of shame, Mae Miller says she never breathed a word of her family’s history, even to her own children, until 2001. Mae’s father, Cain Wall Sr., she says, was born into peonage in St. Helena Parish, La. Census records place the date around 1902, though the family says he is even older. Now in frail health and bed-bound, he married when he was 17 (his wife died in 1984) and by the mid-1930s, the family says, was living across the Mississippi border in Gillsburg, working the fields for white families who lived near each other or attended the same church—the Walls (a common name in the region), the McDaniels and, mostly, the Gordons.

While blacks in nearby towns like Liberty, Miss., attended school, owned businesses and protested Jim Crow laws that denied them civil rights, life in the countryside was a very different matter. The Walls had no electricity, phone or radio. Trips to town, to visit relatives, even to church, were forbidden. Once during World War II, according to the family, Cain Sr. escaped from the Gordon farm. Within two hours he was picked up by two white men; they said they were taking him to a military recruiting station in Jackson, but immediately returned him to the farm. The Amite County school district, where Gillsburg sits, records the six oldest children being enrolled in the fall of 1951—but none of them recall going at that time. “I went to school for a little while in the seventh grade, but I was a lot older than all the other students,” Mae says. “I couldn’t read or write.”

Meals were whatever they could catch—rabbits, birds, fish—and the white family’s leftovers. Beatings with whips or even chains were common, they say, for slacking off or talking back. “The whip would wrap around your body and knock you down,” says Mae’s sister Annie, 67. Mae remembers her father once being beaten so badly that she and her siblings climbed on his fallen body to protect him.

The most crippling violence began when Mae was about 5. She vividly remembers the morning she and her mother went to the Gordon home to clean it. They were met by two men—faces she recognized. One tugged on Mae’s long hair, she recalls. She tried to hide in her mother’s skirt, but he grabbed her and pushed her to the floor. Both she and her mother were raped that morning. “I remember a white woman there saying, ‘Oh no, not her, she’s just a yearling,'” Mae says. “But they just kept on and on.” Mae says her mother begged the men to spare her daughter, and a white women cleaned her up after the attack. That was the first of numerous times she was raped, she says. “They told me, ‘If you go down there and tell Ol’ Cain, we will kill him before the morning.’ I knew there wasn’t anyone who could help me.”

All these years later, it’s impossible to prove Mae’s recollections. There is no legal documentation of the atrocities she describes. “Back then, we did what we had to do to live,” says Mae. “We thought everyone was in the same fix.” When contacted today, a member of the Gordon family has vastly different recollections of that era. Durwood Gordon, 63, a retired propane truck driver now living in McComb, Miss., recalls the family worked for his uncle Willie, a dairy farmer who died in the ’50s, and cousin William Gordon, who was 84 when he died in 1991. “I just remember [Cain Sr.] was a jolly type, smiling every time I saw him,” says Durwood, who was younger than 12 when the Walls worked there. To him, the rape charge is unbelievable. “No way, knowing my uncle the way I do,” Durwood says. “I knew him to be good people, good folks, Christian.”

The Walls finally found freedom in 1961, while working for another family in Kentwood, La. Mae, about 18, refused one morning to clean the house. After the owner threatened to kill her, she ran away. “I don’t know what got into me,” she says. “I remember thinking they’re just going to have to kill me today, because I’m not doing this anymore.” The furious white farmer kicked her whole family off his land.

Not knowing where else to go, most of the Walls stayed near Kentwood. Mae got her first paying job, working in a restaurant for a white lady. “I kept waiting for her to be mean, but she treated me well,” she says. But her past left scars she couldn’t run from. Around 1963, she married Wallace Miller, a construction worker, and wanted to start a family. But a doctor told her that her reproductive system had been damaged, likely from the rapes. Devastated, Mae eventually adopted four children.

Well into her 30s, Mae went back to school and learned to read and write. She became a glass-cutter in the 1970s, a job she held for 20 years. “I started out at a dollar an hour but it seemed like a million to me,” she says with a smile. After her house burned down in 1995 and an injury prevented her from working, she was homeless until 2003. But Mae began cleaning houses and rebounded: With the help of a real estate agent whose office she cleaned, she bought her current house with no money down.