The Scottish village of Firhall, on the outskirts of Nairn on the Moray Firth, is spotless and smart, a cluster of detached houses overlooking a lake. There is no fence between me and the deep water, and nothing to divide one garden from the next. There are no skateboards here, either. No footballs. No noise. It takes a while, but the penny drops: Firhall is the first and only village in Britain from which children are banned, and it shows.

To buy a property in the village, you have to be 45-plus with no dependent family in tow, and you must sign a contract agreeing not to sell property on to those with children. Firhall has, predictably, attracted criticism. "I am writing about a village where children are banned," I tell a friend with young children. "Oh yes, there are places like that in America," she says, "neighbourhoods where no black people are allowed." "What next?" says another, "Villages for women only?"

I liked the idea of a child-free community, and was curious about how it has worked so far. The odd time I have criticised smug parents and their badly behaved children, I have been subjected to scores of emails telling me I am heartless and a child-hater. So I arrive at Firhall hoping to witness the liberation from such ideals for at least a few like-minded folk.

Grandchildren and other young visitors are welcome, but for three weeks maximum. When Firhall was under construction, the brainchild of Caledonian Retreats, a group of chartered architects and surveyors impressed by the success of such lifestyle villages in other countries, buyers were promised a play area for children visiting. Almost three years after it opened in April 2003, it still has not materialised, but no one seems to mind.

In the US, adult-only neighbourhoods have set the scene for other exclusive communities. The first gated village exclusively for lesbians and gays is being built in Florida. But opinion is divided. Is this about the rights of those wishing to live among "their own", or merely a ghettoisation of special-interest groups, which smacks of prejudice and discrimination?

Most people I spoke to at Firhall love living without children around, and are not hesitant in telling you why. "I am sick of having conversations with parents about how funny and clever their kids are," Ivy, a 55-year-old resident, tells me. "I don't think I noticed how much people go on about their kids until I started living among people who don't. As soon as I moved in I felt I belonged," she says. "I am not judged for having no husband or children, or seen as an oddity."

Ivy moved to the village from the west of Scotland and does not regret the move. "I have nothing against children," she says, "but their parents drive me bonkers! I could not bear any more stories about school catchment areas."

Lesley Anne Fraser of Stronachs estate agent, which markets the Firhall properties, is certain that the ban on children is the major incentive for most of those who move to the village. "There is no child-catcher like in Chitty Chitty Bang Bang," she laughs, "but there is also no noise from ghetto blasters, no having to avoid footballs on the pavements."

"It's like Britain was 30 years ago, where you live among considerate and polite neighbours. If a stranger comes into the village, everyone will take notice," says Eden Guisley, the chair of the Firhall Residents Association.

Some residents have children and grandchildren of their own, but feel that they have done their bit and now want to be free of the problems that living among them can bring. "Everywhere you go today you are expected to pander to the needs of children," one home-owner, who has asked not to be named, tells me. "They are noisy, messy and destructive, but try and complain to the parents, and nine times out of 10 you will make yourself an enemy," she says. "I have to put up with badly behaved children in restaurants and parks. I want my neighbourhood to be free of that."

There have been legal challenges to child-free communities in the US. In 1977, a couple was forced out of their Florida condominium after having a baby. The unsuccessful age discrimination suit went as far as the supreme court. There have been other successful cases since then, but none that conclude that living without children should be seen as discrimination. In the UK there are those who believe no one has the right to exclude children from any neighbourhood.

Carolyn Hamilton, the director of the Children's Legal Centre, is adamant that such communities should be challengeable under the Human Rights Act. "If nothing else, it perpetuates the stereotype of children as nuisances and criminals."

On my way to Firhall from Nairn, home to three different schools and an abundance of children of all ages, I chat to 16-year-old Roshanna. "I have heard of Firhall," she tells me, "but had no idea that children were banned. But why would we want to go there?"

Not everyone at Firhall feels they made the right decision in moving to a child-free neighbourhood. Ann, a 49-year-old from the north of England, has lived in the village for two years but recently put her house up for sale. "I came here when I finally accepted I could never have children," she tells me, "thinking it would ease the pain of being around lots of crying babies. But if anything, it has made it worse. I feel the loss more acutely."

Ann miscarried four times in seven years while trying for a baby and eventually paid for three lots of unsuccessful IVF treatment by remortgaging the house in England, and had to sell to downsize. "I chose Firhall after seeing a news report about 'shock, horror', the village where children are banned. I suppose I felt bitter, and just wanted to get away from being reminded what I didn't have, so we came here."

Since the move to Firhall, Ann thinks both she and her partner would feel better facing up to the real world, and plan to move to Glasgow where they have nephews and a niece of school age. "Maybe someone will ask us to be godparents," she says.

The "no children" rule is the primary reason people move here, but, Eden tells me, they also like the fact that "there are other rules too - at least 20, in fact." She begins to rattle them off: no more than three adults to each house; no more than one cat and one dog; no starting car engines after 9pm or before 7am; no running businesses from the premises; no hanging washing outside; no commercial vehicles parked in the village ...

In her office at Firhall House, the mansion that is the centrepiece of the village and residents' community centre, I meet Sharon Macleod, the village lettings manager and troubleshooter. She deals with any complaints from residents, but says she has never heard one about noisy neighbours. "No one can hide behind bad behaviour of children. If there is noise at night, or mess on the grounds, the adults have to take responsibility."

I am invited in for tea with Marie and David, who have lived in the village for two years. They came from Merseyside to be near "like-minded people", in a community where "children are not seen as the centre of the universe".

"Children and noise go together," says David. "It is heaven sitting out with a glass of wine, not hearing kids screaming and banging the fence with footballs. And there is no litter, because it tends to be kids who drop it."

Jimmy and Maureen Greig, both in their early 60s, moved to Firhall from Glasgow to "make new friends, and have a bit of peace". "There comes a time," says Maureen, "when you want to live without children."

I realise as I am leaving Firhall that for me, the absence of children does not necessarily appeal. It feels too contrived, too quiet and rule-bound. Living in my ivory tower in a neighbourhood where, so far, I have been lucky enough to have well-behaved children playing in my street, an adult-only environment is not something I hanker after.

So will the Firhall concept take off elsewhere in the UK? Certainly there are no regrets from the residents. But occasionally things can get a little dull. "There is no gossip," says Maureen. No scandal. That is the one thing I miss."