The tour starts at a restaurant just outside Colorado City.

It's called the Merry Wives Café, which is ironic because there's nothing very merry about this remote and windswept town on the border between Arizona and Utah, since local authorities haven't got round to re-legalising the sale of alcohol following the end of Prohibition in 1933. But if we're going to split hairs, the people who named the café deserve credit for at least one thing: the local community does indeed contain an awful lot of wives.

Our guide, Richard Holm, should know. He's been married three times: first, to his childhood sweetheart, Shauna, in 1971; then to a much younger woman, Lorena, in 1985; and finally, to a girl called Alice, in 1996. Between them, these lucky ladies bore him 17 children. Oh, and in keeping with local custom, all three were married to him at exactly the same time. That's because Colorado City is a polygamist community, and for many years, Richard was one of its most enthusiastic members.

Today, things are different. The 57-year-old still lives in the town, with its vast, 30-room homes holding extended poly-families, but he's no longer a member of the Mormon sect which has safeguarded the tradition of plural marriage there since the 1930s. He's abandoned organised religion, divorced all three wives, and recently launched a new career as a tour guide, taking bus parties of curious visitors around his hometown, on a trip he calls "The Polygamy Experience."

"Tourism is such a big industry in the region, with Vegas a few hours away and Zion National Park around the corner, but this has always been a closed community," he explains. "I want to change that. I want people to stop in Colorado City: to look round, spend a bit of money, and try to understand. Locals have their doubts: they say they don't want to be treated like monkeys in a zoo, and I understand that. But I aim to be respectful."

Richard packs small tour parties into his car, and transports larger groups around the dusty streets of Colorado City in a 30-seater bus. The tour lasts four hours, costs $70 each, and consists of a compelling mixture of local history and Richard's well-worn personal anecdotes, interspersed with faintly-bawdy jokes. "Why the prairie dresses and long braids?" reads a leaflet advertising the tour, illustrated with pictures of local women in their traditional outfits. "Why take more than one wife?" Find out for yourself, from guides who "have lived and loved The Creek" – the name locals give the town.

Our trip this week took in the town cemetery, where "prophets" are buried, and its parks framed by red cliffs, where locals like to hold their massive family picnics (Holm had 34 brothers, 25 sisters, and 11 "mothers"). At the end, we stopped to buy home-made ice cream and cheese from the local dairy.

You may have heard of Colorado City because it's home to the Fundamentalist Church of Jesus Christ of Latter-Day Saints (FLDS), which was thrust to the centre of a media firestorm in 2008 when police raided one of its compounds in Texas and took 400 children into care. The kids were returned to their parents, but not before roughly a dozen senior figures connected to the church had been charged with offences including corruption, fraud, and child abuse. The church's leader, Warren Jeffs, is a somewhat notorious figure. To followers, he's a prophet with a hotline to God; to outsiders he's a silver-tongued cult leader and fraud, who marries underage girls off to his elderly cronies. Either way, he spent time on the run from the FBI, who added him to the "America's Most Wanted" list, before he was arrested and convicted as an accessory to child rape. He's serving 10 years to life in Utah State Prison. Last month, two of his closest associates were jailed on similar charges.

Against this backdrop, Colorado City views outsiders with suspicion. Residents take exception to being gawped at by tour parties, and dislike being photographed in their traditional outfits: prairie-style dresses, big hair, and no make-up for the ladies to symbolise modesty. Richard is adamant his tours will reinvigorate the local economy and increase understanding of a vilified way of life. "At its best, polygamy teaches self restraint and love, and self-discipline," he said. "It teaches you to share and to build a community. I do think it can go too far: the underage marriages, and the abuse and fraud that takes place under the cover of secrecy and darkness, that's got to stop. But consenting adults should be allowed to live how they like within the umbrella of decency and the law, and if they want, have more than one wife."

Ever wondered about the nuts and bolts of daily life in a plural marriage? Richard says domestic arrangements are organised in a way familiar to viewers of the TV series Big Love: wives "share" a husband by sleeping with him in rotation, and taking joint responsibility for bringing up each others' children. These extended families inhabit Colorado City's enormous homes, generally hidden behind high fences, walls, and "keep out" signs.

The tradition of polygamy began with the Mormon Church, whose founder Joseph Smith decided that, to make it into heaven, a man needed to have at least three wives. It was abandoned by the organisation in the late 19th century, after Smith's death, so Utah could be allowed to join the United States. At the time, Mormon elders claimed to have experienced a "revelation" in which God told them to revert to normal, exclusive marriages. But some followers refused to buy that. They formed conservative sects in remote regions of Utah, Arizona and Idaho, and carried on practising polygamy. An estimated 37,000 remain. The FLDS is one of the most successful of the breakaway sects and Colorado City, with 8,000 residents, has grown into the biggest polygamist town in the US and possibly the world. The church, which controls all the town's land via a trust, has built its own hospital, a zoo, and several schools. One prominent local runs a well-known luxury hotel chain.

Like most Mormon sects, the FLDS is headed by a "prophet," or elder who has declared himself in direct contact with the Heavenly Father. He controls the organisation's finances, collected via a monthly levy or "tithe" on members. More importantly, he's also put in charge of deciding when a man ought to be allowed to acquire new wives. "Each month, I would go to see the Prophet and pay him my tithes," said Richard. "I'd give at least 10 per cent of my income. I did well for myself in the construction business, so that was a decent sum of money. One day, over a decade after I'd taken my first wife, the Prophet said he'd had a revelation and that I should get married to Lorena, who I'd never met. That's how I became a polygamist."

The flip side of the enormous power wielded by church elders is of course that it can be abused. Richard experienced this first hand when the FLDS prophet Leroy Johnson died, to be replaced first by an elderly man called Rulon Jeffs, and later by his son, the now-notorious Warren, who took charge in 2002, and promptly added his father's several dozen wives to his harem of over 100. Warren spent millions of dollars of the church's money supporting his extravagant lifestyle and building a new community, called the Yearning for Zion ranch, at Eldorado in Texas, where he moved with 1,000 followers.

During childhood, Warren had often clashed with Richard; once in power, he decided to get even, ejecting him from the FLDS in 2003. "One by one, my wives were called to see Warren and told he'd had a revelation from God, telling him I was wicked and sinful, and they should leave me and marry someone else," said Richard. "By November that year, they'd gone, taking my children with them." Richard spent $200,000 on a court battle to be given access to his kids (which he won). He was evicted from his home by the church (a move he overturned). And his construction firm collapsed when one of his ex-wives, on the advice of Jeffs, remarried his brother and business partner, Edson. In all, he says, it cost him $2m.

Church followers are banned from having television sets, and their internet access is closely monitored, since the local web provider is a company run by a church elder. They live, for the most part, in closeted isolation. Despite this, and because of heroic efforts by people like Richard, Colorado City is slowly becoming more open. Members of the FLDS, who number at least 6,000 locals, largely believe Jeffs to be a martyr who has fallen victim to religious persecution. "His defence," said Richard, "is that no court or government has the right to tell God's prophet what to do." But others have grown sceptical of his reign, during which the FBI believes $96m was siphoned from the local community. A feature-length documentary about the FLDS, Sons of Perdition, is attracting buzz on the film festival circuit, and hits cinemas this summer. It tells the story of young men expelled as teenagers to prevent them attempting to marry girls who church elders would rather have for themselves. And with Jeffs in prison, and many associates awaiting trial, the FLDS has abandoned some of its excesses.

From the church itself, however, there remains a deafening silence. When Richard first announced that he was launching his tours, shortly before Christmas, FLDS spokesman Willie Jessop complained to a local newspaper that "[Visitors] want to come into the community like it's a spectacle, when for us, it's like the circus is coming to town. We hope people have more of a life than to be suckered into that sort of scam."