The greek scholar Diodorus Siculus wrote of the mythical double-sexed Hermaphroditus: "Some say [he] is a god and ... has a body which is beautiful and delicate like that of a woman, but has the masculine quality and vigour of a man. But there are some who declare that such creatures of two sexes are monstrosities, and coming rarely into the world as they do, have the quality of presaging the future, sometimes for evil and sometimes for good."

Throughout history, those born with both male and female physical features have been beset by society's interpretations of them as freaks (The Lancet branded a British hermaphrodite "a disgusting spectacle" in 1834), prophets (the part-male/part-female character Tiresias in Oedipus Rex and Antigone was a clairvoyant), or both. What they've seldom been allowed to do is just get on with their lives.

Even in modern times, tales of "intersexed individuals", as they are now known, are often tinged with melancholy. In 1998, a Sunday newspaper reported on the "desperately lonely existence" of Linda Roberts, who was "spat at and stoned" in a village in Snowdonia for her androgynous appearance.

Last year, the South African track athlete Caster Semenya, then just 18, was subjected to chromosome testing and a humiliating media furore when it was discovered she had no ovaries or uterus, as well as unusually high testosterone levels. The International Association of Athletic Federations remains undecided on whether she is "really a woman", and has suspended her from competing in major sporting events, including last weekend's World Indoor Championships in Qatar, until a verdict is reached.

Semenya has no desire to be a pioneer for the intersex community; she simply wants to concentrate on her running. But there are countless others who are calling for increased recognition of their status, as well as for a moratorium on the practice of "sex reassignment surgery" of intersexed infants.

Intersexuality presents itself in numerous different forms, occurring both in individuals with a standard karyotype (the chromosomal make-up of the body's cells) of 46, XX (female) or 46, XY (male), as well as in those with more unusual karyotype combinations such as 47, XXY or 47, XYY. In one of the most common conditions, Congenital Adrenal Hyperplasia (CAH), overactive adrenal glands can cause a female foetus with XX chromosomes to "over-virilise", developing anything from a large clitoris to a fully formed phallus, while the rarer Androgen Insensitivity Syndrome (AIS) can cause a male foetus with XY chromosomes to "under-virilise", developing only a partially formed penis and testes.

Until recently, the overwhelming response among doctors was to surgically "correct" a baby's ambiguous genitalia on the grounds that he/she would, in later life, be stigmatised by these unconventional appendages. From the 1960s, it became common practice to trim down an enlarged clitoris, and to fashion a malformed penis into a vagina. The line among surgeons was allegedly: "It's easier to dig a hole than build a pole."

Intersexed babies with XY chromosomes have therefore frequently been "reassigned" as female, with parents advised to raise them as girls, and oestrogen pills administered to induce female puberty. This is largely due to the hugely influential 1960s "optimal gender policy" of psychologist John Money, and his famous assertion that nurture could override nature.

The prevalence of corrective surgery is in part responsible for our general ignorance about intersexuality, which is far more widespread than most of us realise; the number of live births displaying "genital dimorphism" is estimated at approximately one in every 2,000. That means there could be as many as 30,000 intersexed people currently living in Britain, a figure that becomes even greater when taking into account all those who only discover their condition at puberty, or when they try to have children. As the renowned professor of neurology and intersex expert Dr Milton Diamond puts it: "Nature loves variety. Unfortunately, society hates it."

"Our constant pursuit of perfection has left many children infertile, with their gender identity stolen," argues Dr Jay Hayes-Light, a specialist in child mental health and the director of the campaigning organisation the UK Intersex Association (UKIA), which formed in 2000. "There's this fear that if we have women with large clitorises and men with small penises, it'll be the end of civilisation as we know it. In fact, the individuals who end up most damaged are those who are surgically altered, without their permission, to suit someone else's agenda."

Why? Because as well as the physical downsides – "reducing a clitoris carries the risk of permanent loss of sensitivity, while reducing a phallus and removing the testes destroys the child's fertility" – Hayes-Light insists that gender cannot simply be reconfigured through surgery and hormone treatment.

His own studies of individuals who were reassigned as infants, some of whom have eventually transitioned back to their original gender, and some of whom have committed or attempted suicide, have convinced him that "sex is between the legs while gender is between the ears. Most people are 'hard-wired' to a gender identity, whether this is male, female or something in between."

Hayes-Light, now 48, was himself born with an extremely rare intersex condition, 5-Alpha-Reductase Deficiency (5-ARD), in which the chromosomes are XY, but the body is unable to convert testosterone to dihydrotestosterone, which is necessary for the formation of full male genitalia. "This was the early 1960s; I was a lab rat, a medical curiosity. What saved me from being poked and prodded – and operated on – was the fact that my mother was a doctor."

Advised that he would be "a very happy little girl", Hayes-Light's mother nevertheless declined surgery for her son. But she agreed to try raising him as female, and did so until he was 10. "I was then asked if I wanted to take oestrogen tablets, to push my body through a female puberty," he remembers. "I refused." Despite this troubled and confusing start to life, Hayes-Light considers himself one of the lucky ones: "I shudder to think what I'd be like now if I'd been forced to transition."

Adele Addams wasn't given such a choice. Born in the late 1970s, she has Klinefelter's Syndrome, so is XXY; chromosomally neither male nor female, in other words. Her parents were encouraged to assign her as male, and surgery was performed immediately. In her mind, this was "the wrong decision". After a difficult childhood and adolescence – "I eventually went into care, as my family couldn't cope with what was supposed to be a boy but looked and sounded more like a girl" – she began living as a gay man. Yet this didn't feel right either, and three years ago, after "a lot of fighting", she was granted NHS surgery to make her body female. Now 31, she finally feels that "it's all come good", and is passionate about empowering others to define their own gender.

Last year she set up Project Silverfish, a support service for intersex and transgender people, whom she describes as "the most marginalised minority community in London". Named after her friend, the transgender DJ and music producer Alex Silverfish, who committed suicide in 2008 following years of harassment, the project offers its users practical advice and advocacy, and has just launched its first therapeutic support sessions. It also delivers training to charities and social services.

Addams, who last December won a Home Office Community Engagement Award for her work, explains that while transgenderism and intersexuality are not the same (the former is not a medical condition), "it's not uncommon for trans people to discover later in life that they're intersex".

This was what happened to Alexandra Tovey, aged 37. From an early age, she felt she was "born with a female brain, but a male body", yet her intersex condition – Partial Androgen Insensitivity Syndrome (PAIS) – was only diagnosed last year. She'd already been living as a woman for several years, and the news, she says, "made things make more sense".

Now enjoying "being the 16-year-old girl I wasn't allowed to be when I really was 16", and awaiting a full sex-change operation, Tovey, a singer-songwriter, expresses herself through music. She's released 18 albums since 1992, and many of her lyrics reflect her experiences as both a trans and intersexed person.

The actress and playwright Sarah Leaver is also using art "to expose what lives between the lines". Her play, Memoirs of a Hermaphrodite, currently running at London's Oval House Theatre, tells the true story of Herculine Barbin, a 19th-century intersexed Parisian whose musings were published by the French philosopher-sociologist Michel Foucault in 1980. Barbin began life as a girl, but faced misunderstanding and contempt when she fell in love with another woman, and was later discovered to have both male and female genitalia. Forced to become a man, she descended into depression and poverty, and died in tragic circumstances.

Thankfully, Leaver's own tale is far happier. "I've always felt in between the genders," she says. "As a kid I wore boy's pants, played football, and ran around with my top off." Having supportive parents meant this was rarely an issue, and it was only in her teens and twenties that she started to question her identity: "I didn't feel I was in the wrong body, but I knew there was something that made me different."

Six years ago, after watching a documentary on intersexuality, she asked her GP to check her medical history. It transpired that an operation she'd had as a toddler in 1977 to remove a "hernia" had in fact been to remove a male gonad – but neither she nor her parents had ever been informed. "It was like finding the missing piece in a jigsaw," she recalls. "Part of me was relieved, and part of me was really angry. Why had the doctors hidden this from me?"

Despite her anger, Leaver, who is now 34, regards herself as pretty fortunate. Her condition – she hasn't yet sought a specific diagnosis – is at the mild end of the intersex spectrum, as she has XX chromosomes, ovaries and periods, and has experienced few health problems.

Through the play, she's turned her frustration into "a force for change", and hopes others will do the same: "More and more people are speaking out about being intersexed, and the time feels right for celebrating and embracing our differences; not hiding them away."

"We need to be able to live as ourselves," agrees Addams, "which means making room for more than two genders." Tovey echoes this sentiment: "I've always felt very female, but I know others who identify with both sides, who feel 'in between'. They need acceptance, too."

Hayes-Light, though respectful of the wishes of those who want to be considered "gender-neutral", stresses that they are a minority: "Although some physically intersex people choose not to identify as either male or female, most do identify as either one or the other."

When it comes to operating, all four of these case histories concur that modifying infants' genitals for anything other than a medical necessity is wrong. And it appears that even the medical experts have started to come round to this view. In 2001, the British Association of Paediatric Surgeons (BAPS) recommended that babies with ambiguous genitals should not be given corrective surgery, and should be left to decide their own gender eventually.

Things seem to be moving on in other fields too. "In the 10 years since UKIA was founded, we've started to see far fewer cases involving secrecy and cover-up [such as that experienced by Leaver]," says Hayes-Light. "We've also succeeded in getting intersex included in the 2004 Gender Recognition Act. And we've seen major changes in social attitudes, with more balanced media reporting, too."

Yet there's still a long way to go. "Society is becoming more open-minded, but most people still haven't heard of the word 'intersex', and gender variance remains the last taboo in our culture," maintains Addams. Despite the recommendation of BAPS, surgery on intersexed babies still goes on, both in Britain and many other countries.

Leaver looks forward to a time "when intersexed people are no longer viewed with disdain or pity, and the world doesn't cling so much to 'male' and 'female' ".

No one can predict the future, but it does seem as though our culture is becoming, albeit slowly, less rigid about gender roles, and more accepting of unconventionality in general. It can only be hoped that it's a sign that we're beginning to respect, and even learn from, the individuals we would once have relegated to circus sideshows, and later to operating theatres.

As the writer and psychotherapist Amy Bloom sums up: "Not monsters, nor marvels, nor battering rams for gender theory, people born intersexed have given the rest of the world an opportunity to think more about the odd significance we give to gender, about the elusive nature of truth, about the understandable, sometimes dangerous human yearning for simplicity – and we might, in return offer them medical care only when they need it, and a little common sense and civilised embrace when they don't."

Or, in the simple, succinct words of Dr Milton Diamond: "Let's see if we can change society, not nature."