Recently I wrote about the many problems polygamy tends to cause across the world, including high crime rates resulting from young men confined to singledom because older men are hoarding wives, and the subjugation of teenage girls forced to marry because there simply aren’t enough women to go around. With few exceptions, such as among India’s Toda people, a pastoral tribe in which one woman sometimes married several brothers, polygamy almost always expresses itself as the marriage of one man to several women. In America, polygamy is associated with fundamentalist Mormon culture, and more specifically with its domineering leader Warren Jeffs and his sexual abuse of underage girls.

Historically, though, there’s been an exception to the rule about plural marriage being bad for women. Polyamory, in which people openly take on multiple relationships, sometimes in the context of group marriage, has a radically different history. Nearly as marginal on the left wing of our culture as polygamy is on the right, modern-day polyamory is intertwined with the rise of feminism, and its roots go back to the ’40s— the 1840s. It’s hard to believe, but during the heart of the Victorian era, during a time when chastity was the rule, divorce was unheard of and petticoats were unmentionables, the most radical American women renounced monogamy as an instrument of their servility. A progressive attitude toward gender roles continues in the modern-day polyamory movement, which has been shepherded by women writers, historians, and organizers.

From the late 1840s to the late 1870s, under the leadership of a charismatic Christian minister named John Noyes, the Oneida commune in upstate New York conducted an experiment in promiscuity known as complex marriage. Noyes believed that sex was a kind of worship, and that in order to live without sin, men and women had to be free to worship all over the place with whoever they wanted. About 300 people lived at Oneida, and they were all considered married to one another. Noyes had radical and sometimes abhorrent ideas about sex; he tried to breed a better class of humans through eugenics, and he thought incest was just fine. (At various points he had sex with his niece, and possibly his sister.)

Despite its many faults, though, the system of complex marriage at Oneida amounted to remarkable progress for the women who lived there. Older women were responsible for teaching young men exactly how to practice spiritual sex. Men were responsible for birth control by resisting orgasms during intercourse. Oneidan women generally had sex only with whom they wanted, which, as Oneida historian Spencer Klaw points out, “could not be said of numberless married women in the world outside.” In her diary, Noyes’ niece Tirzah Miller wrote of the last, bittersweet days of complex marriage. She made love to one man, and then another, and then, while fetching a lemon for one of her lovers, came upon a third. “There seemed a subtle fire between us,” she wrote, and slipped into the bathroom with him.

This ethic of free love, honed during Noyes’ time, continued into the next century, albeit only on the fringes of society. Free love rejected the tyranny of conventional marriage, and particularly how it limited women’s lives to child-bearing, household drudgery, legal powerlessness, and, often enough, loveless sex. As historian Christine Stansell documents in her book American Moderns, bohemians paired the free-love ethic with the fight for birth control and the idea of women’s sexual self-determination. In practice, that made for some interesting romantic dynamics. The anarchist Emma Goldman lived with her boyfriend and another couple, and the four of them made love at the same time as they made political trouble. The writers Neith Boyce and Hutchins Hopgood were married to each other but compared notes on their various lovers. (“Tell me you love me and also tell me about the flirtations you are having,” Hutchins wrote Boyce.) In England, the intellectuals of the Bloomsbury group contorted themselves into all sorts of tortured love triangles.

In his book, The Lifestyle: A Look at the Erotic Rites of Swingers, journalist Terry Gould draws a direct line from Oneida to the bohemians, through the free love of the beatnik and hippie movements and into the modern-day polyamorous community. There is a thread of idealism that runs through all of it, he writes; polyamory has historically been as much about remaking one’s own little corner of the world as it has been about sex. Gould contrasts polyamory with swinging, which he traces to the most unlikely of sources—World War II era fighter pilots and their wives. Swinging thus has a more masculine and more conservative history, whereas polyamorists have tended to be more utopian, more New Age, and yes, more feminist.

In the 1970s, during what sociologist Elisabeth Sheff calls the second wave of polyamory, fringe groups around the country experimented with non-monogamy. A San Francisco-based commune called Kerista, founded by a man who called himself Jud the Prophet , consisted of three large group marriages, in which sleeping schedules were rotated regularly to keep intimacy evenly distributed. Although years later some former members would describe Kerista as a cult dominated by a charismatic man (not unlike John Noyes at Oneida), when the group was active it described itself as a product of the women’s movement. Its 1979 handbook mandated egalitarianism and required that members care for the commune’s children in “non-sexist parental roles.”

During the ’90s, the Internet sparked a third wave of polyamory, after AIDS had driven it underground during the ’80s. A Usenet newsgroup called alt.polyamory helped build a community, and a woman calling herself Morning Glory Zell, member of a “neo-Pagan” organization called the Church of All Worlds, helped popularize the term in an article called “A Bouquet of Lovers.” In more recent years, polyamory has mainstreamed somewhat, becoming fodder for features in Newsweek and on ABC’s Nightline. MTV did a True Life documentary on polyamorous young people, books like The Ethical Slut explored the topic, and Dan Savage continues to champion non-monogamy. Polyamory is no longer primarily identified with pagans and prophets.

In the most crunchy, West Coast communities, group marriages and open marriages are common enough that people can talk about being “poly” without having to explain what that is, says Sheff, a Georgia State University professor who is working on a book about polyamory. In her research, Sheff has even come across an area in Seattle populated by large polyamorous families: “You’ve heard of gayborhoods? This is the first poly-neighborhood I’ve heard of.”

Women are in many ways the driving force behind polyamory as a movement these days, having been integral in founding its organizations and documenting its history. As Jessica Bennett pointed out on Slate’s XX Factor back in 2009, the first books on the movement were written by women. Gould writes that a sizable number of polyamorous households consist of more men than women, the opposite of how polygamy typically expresses itself.

Group marriages are hardly dream harems for the women at the helm. The issues are those of monogamous relationships, but magnified considerably. In a marriage of five, who pays the bulk of the rent? Who empties the dishwasher? Polyamorous conventions tend to offer practical advice. Along with lessons in dancing in “Goddess” style, they have sessions on the logistics of large families sharing housing, and how to keep the peace between one’s various partners. Talking is the main tool of the polyamorists; they negotiate their time, their tasks, their emotional landmines.

In one triad, a female sexologist lives with her two boyfriends in Topanga, Calif. One boyfriend, with whom she has a son, works as a furniture designer, while her other boyfriend stays home and takes care of the baby. “Jealousy comes up,” the sexologist says, but they deal with it through “communication.”

On the other hand, there is something to be said for being the center of a polyamorous universe. This is the position of Terisa Greenan, an actress who lives with two men in Seattle. They all date outside the household, but this triad is their primary relationship, and both men appear deeply in love with Greenan.

As she puts it in one video with a grin, “You get something different from each of them.”