The line for which the Supreme Court Justice Oliver Wendell Holmes, Jr., is best known is also his ugliest: “Three generations of imbeciles are enough.” He was referring to a woman named Carrie Buck, her mother, and her daughter, in the case of Buck v. Bell, from 1927, which held that it was constitutional for a state to forcibly sterilize Carrie, a poor white woman who was deemed “feeble-minded,” for the greater good of promoting social welfare. Summing up the views of the eugenics movement, an avid cause of progressive reformers and intellectual élites at the time, Holmes explained, “It is better for all the world if, instead of waiting to execute degenerate offspring for crime or to let them starve for their imbecility, society can prevent those who are manifestly unfit from continuing their kind.” The opinion was joined by all of the Justices save for one, Pierce Butler, who was the Court’s lone Catholic at the time. Catholics, a population that many Protestants wished would reproduce less, vocally opposed eugenic sterilization as being an unnatural interference with procreation.

Buck v. Bell has never been overturned. In fact, the case was cited approvingly in Roe v. Wade, in 1973, for the proposition that one does not have “an unlimited right to do with one’s body as one pleases,” which meant that a state was constitutionally permitted to put some limits on the right to abortion announced in Roe. For some, the history of states’ forced sterilization of women—and the Supreme Court’s role in permitting it—might seem a sinister precursor to contemporary efforts to control women’s sexual and reproductive freedom by restricting access to abortion. For others, the history of eugenics in America might instead provoke a very different fear: that progressives today, much as in the past, may wish to prevent the unfortunate and disfavored from being born—this time by allowing individuals to abort certain fetuses, and doing so with a discriminatory tilt.

Concurring in an abortion case in May, Justice Clarence Thomas penned a lengthy and lurid polemic, warning that abortion rights are a form of racist eugenics revivalism. The statute at issue, passed in Indiana and signed into law, in 2016, by Mike Pence, the governor at the time, prohibited providers from performing an abortion if they know that it is sought solely because of the fetus’s race, sex, or disability. The Seventh Circuit invalidated the law as unconstitutional, under the Supreme Court’s precedents, because it was an “undue burden” on the abortion right. The Supreme Court declined to take up the case, expressing no view on its merits, and decided instead to wait until other courts of appeals weigh in on a handful of similar laws from other states. Thomas agreed with the Court’s denial of the petition, but he wrote separately to make clear that laws like Indiana’s “promote a State’s compelling interest in preventing abortion from becoming a tool of modern-day eugenics.”

Thomas connected a series of real dots: the “scientific” belief in black inferiority that informed the early-twentieth-century eugenics movement; the eugenicism espoused by Margaret Sanger, the founder of Planned Parenthood; and Sanger’s promotion of birth control in poor black neighborhoods. Some mid-century arguments for legalizing abortion—including those made by Alan Guttmacher, who went on to succeed Sanger as president of Planned Parenthood—made eugenic appeals, saying that abortion was important to controlling the “quality” of the population. Thomas linked those facts to contemporary statistics on abortion. According to the New York Department of Health, in some areas of New York City, “black children are more likely to be aborted than they are to be born alive—and are up to eight times more likely to aborted than white children in the same area.” Thomas even called up the contested assertion, made by the journalist Stephen Dubner and the economist Steven Levitt in their book “Freakonomics,” that Roe v. Wade led, a generation later, to a massive decline in crime, because the availability of legal abortion meant that many people did not have the unwanted babies whose poor circumstances would have led them to grow up to commit crimes. Thomas discerned in all of this an echo of eugenicists’ racially inflected wish for society to be rid of the “unfit.”

While the eugenics and abortion movements may have disquieting intersections, the notion that abortion rights are the direct heir to our history of eugenic sterilization is unfounded. Nobody is advocating forcible abortion, for eugenic or any other reason. A state forcibly sterilizing women from disfavored groups bears little similarity to a state allowing individuals to make decisions to terminate their own pregnancies—even in cases in which they may do so because of the fetus’s race, sex, or disability. The former eliminated a person’s ability to decide whether to reproduce, whereas the latter enables it.

But it is important to understand that the alarm over abortion as eugenics is a decoy of sorts. A deeper, more troubling argument that is now gathering force is tucked more quietly into Thomas’s invocation of legal anti-discrimination norms. If the right to be free of discrimination on the basis of race, sex, or disability can be made relevant to a fetus, then fetuses are figured as entities with anti-discrimination rights—like people. This move imbues the fetus with rights that the pregnant person—and, by extension, the abortion provider—might violate. What is really at stake is an idea of fetal personhood.

It is not coincidental that in the same case, last month, the Court upheld part of the Indiana law, which prohibited abortion providers from disposing of fetal remains as they would surgical waste. Keeping the law in place, the Court reasoned that how fetal remains are disposed of after abortion doesn’t affect access to the abortion itself. But it does transform cultural practices surrounding the treatment of fetuses, through gestures that suggest they are person-like entities, and point at their rights. Indeed, in defending the law, Indiana asserted an interest in the “humane and dignified disposal of human remains.”

Writing in 1990, the constitutional scholar Laurence Tribe called abortion “the clash of absolutes,” referring to the clash between the fetus’s development and the pregnant person’s liberty. On one side, the belief that a fetus is a human being would mean that abortion is a form of murder, which makes the idea that it is a woman’s “choice” callous or nonsensical. On the other side, the belief that the abortion decision belongs in the domain of individual autonomy rests on the assumption that, whatever it is, abortion is not the killing of a human being. Tribe observed that “solutions that split the difference—denying some fetuses life and some women liberty—hardly offer a solution.” But splitting that difference has been our legal solution for half a century. During this time, the interest of neither the fetus nor the pregnant woman has been treated as absolute.

In Roe v. Wade, the Supreme Court explicitly refused to “resolve the difficult question of when life begins.” From there, over multiple decisions a scheme emerged of weighing the state’s interest in protecting the fetus against the woman’s limited right to abortion, which lessens through the pregnancy in relation to the fetus’s increasing age. In holding, at different times, that states may not act on the belief that human life begins at conception or plant a “substantial obstacle” in the path of someone who wishes to abort a fetus before “viability”—a line that is constantly shifting along with improvements in medical technology—the Court has discreetly drawn its own limits around the big question it said it wasn’t resolving.