People like stories about heroes rescuing the exploited and enslaved. Films like Amistad provide thrilling narratives, in which good white people like John Quincy Adams defy convention to free the unjustly chained from bondage. Similarly, journalist Nicholas Kristof famously live-tweeted a police raid on a Cambodian brothel, supposedly so that his followers could experience the freeing of victims of human trafficking in real time. Awesome heroes, evil antagonists, dramatic, instantaneous liberation: It’s easy to see these stories’ appeal.

Some 15 years ago, Austin Choi-Fitzpatrick was involved in one of those supposedly heroic narratives. In 2001, Choi-Fitzpatrick was horrified by stories of the exploitation and trafficking of girls in India. He joined a nonprofit and started taking part in sting operations, trying to get young girls out of brothels. What he discovered, however, was that the story of salvation and emancipation didn’t work out in real life. The girls he helped rescue were mostly placed in poorly run, underfunded orphanages. Many of them ran away—often back to brothels. Choi-Fitzpatrick realized he wasn’t helping. Indeed, in many ways, rescue groups and exploiters mirrored each other. Neither asked those in bondage what they needed, or what they wanted. The rescuers did not, and in many cases did not even try, to give enslaved people agency.

Choi-Fitzpatrick is now an assistant professor at the Kroc School of Peace studies at the University of San Diego. His recent book, What Slaveholders Think: How Contemporary Perpetrators Rationalize What They Do, takes an innovative approach to analyzing and understanding the often-complicated reality of modern-day slavery and exploitation. Over a year and a half, Choi-Fitzpatrick interviewed 300 people, including slaveholders in India using exploited labor (mostly in agriculture), people from the slaveholder’s communities, and people held in, or previously held in, slavery. Choi-Fitzpatrick also interviewed numerous people from the slaveholder’s communities, as well as people held in, or previously held in, slavery.

Through his research, Choi has helped illuminate how anti-slavery and anti-trafficking advocates and slaveholders alike often treat the disempowered with paternalistic condescension. And he ended up with a unique perspective on what liberation means—and on the importance of asking those in bondage what liberation actually means to them, instead of relying on the people doing the rescuing (or slaveholding).

In India, slavery is often accomplished through debt bondage. Poor lower-caste agricultural workers have little access to credit. They do not have money to pay for a wedding or a burial, or even to make emergency purchases of food. Desperate workers can borrow money from business people who specialize in this type of high-risk lending. In return for the emergency cash, the debtors do manual fieldwork for long hours—one of Choi-Fitzpatrick’s informers notes that the advantage of debtors is that you can make them work longer. Over time, workers accrue more debts. Sometimes, because they have little education, they may not be able to calculate when they have worked the debt off.

“People with no resources are taken advantage of by people who have social, economic, and political power,” Choi-Fitzpatrick explains. “Marginalized people end up trapped in cycles of debt that last for years, and in cases I’ve seen end up lasting decades and even generations.”

Slaveholders in India exploit workers and in some cases use violence against them. Nonetheless, Choi-Fitzpatrick’s interviewees did not see themselves as villains. Moreover, their communities, and even in many cases the workers themselves, did not see the slaveholders as villains. Instead, the relationship between slaveholders and the people they exploit is viewed, as it was in the American South, in a context of paternalism.

As one of Choi-Fitzpatrick’s interviewees said, “I am a master. I have employed people and done so much for them. I have fulfilled my duties toward my servant. They should realize this and fulfill their duties as a servant.” Slaveholders think of themselves as “caretakers,” Choy-Fitzpatrick says, and the culture around them generally validates that view.

Slaveholders, in other words, think that they are righteous benefactors helping the weak—just as anti-slavery advocates think they are righteous benefactors helping the weak. To an observer, anti-slavery interventions may seem less harmful—though raids and state orphanages can also be traumatizing and even dangerous when they expose victims to abusive police. The downsides of rescue work can be difficult for the rescuers to understand, points out Choi-Fitzpatrick, because they, like the slaveholders, have “the sense that they’re going to go in and rescue people who are completely exploited, who have no agency, and the only thing standing between them and their horrible lives and true freedom is the act of an intervening savior.”

This kind of paternalist condescension needs to go if anti-slavery groups are to be effective, Choi-Fitzpatrick says. These advocates, according to Choi-Fitzpatrcik, need to ” work with communities and individuals in community so that they are able to be in the forefront of their own emancipation.” In his opinion, successful efforts to help exploited people will likely look more like community organizing or labor organizing than like police raids.

In his book, Choi-Fitzpatrick notes that there is evidence to back up this alternative approach. Non-paternalistic projects, focused on changing social conditions rather than rescue, have been quite effective in India. For example, programs that make food freely available to the poor, or roadwork projects, have reduced the power of slaveholders and given people a way out of debt-bondage. When people don’t need to worry about starving, they don’t need to go into debt to feed their families. When they can get a decent wage for a day’s work, they don’t need to accept a slaveholder’s offer.

As people have more access to resources, they can make more demands, Choi-Fitzpatrick explains. Rather than being saved, they gain the power to demand their own rights, much to the chagrin of their self-styled owners—and sometimes to the chagrin their saviors. Choi-Fitzpatrick notes that in many cases, people in exploitive situations may have different goals than the advocates hoping to help them. For example, their main goal may not be immediate emancipation, but more access to schools for their children. More education, though, can be a first step toward other rights. Slaveholders Choi-Fitzpatrick talked to openly complained that more education weakened their hold on workers.

Slaveholders believe they know what is best for the people they enslave. Ironically, they see themselves as benevolent saviors, much like Nicholas Kristof did during his brothel raid. It’s important to remember that exchanging one paternalism for another isn’t empowerment. “We don’t know what form freedom will take,” Choi-Fitzpatrick says. To find out, he says, we need to ask those who aren’t free.

Update: This story has been updated to give a more accurate representation of the number of people Choi interviewed in India.