Such was the media excitement inspired by the appearance of a vibrator in a late 1990s episode of Sex And The City, one might have thought the device had only just been invented. Any misapprehension is about to be corrected by a new film, Hysteria, which tells the true story of the vibrator's inception. Described by its producers as a Merchant Ivory film with comedy, Hysteria's humour derives chiefly from the surprise of its subject's origins, which are as little known as they are improbable.

The vibrator was, in fact, invented by respectable Victorian doctors, who grew tired of bringing female patients to orgasm using their fingers alone, and so dreamt up a device to do the job for them. Their invention was regarded as a reputable medical instrument – no more improper than a stethoscope – but became wildly popular among Victorian and Edwardian gentlewomen, who soon began buying vibrators for themselves. For its early customers, a vibrator was nothing to be embarrassed about – unlike, it's probably safe to assume, many members of the film's contemporary audience, not to mention some of its stars.

"I've done a lot of 'out there' sexual movies," Maggie Gyllenhaal readily acknowledges, "but this one pushed even my boundaries." Gyllenhaal plays a spirited young Victorian lady, and the love interest of the doctor who invents the vibrator, but admits, "I just think there is something inherently embarrassing about a vibrator. It's not something most people say they've got; nobody talks about that, it's still a secret kind of thing. So it's very difficult," she adds, breaking into a laugh, "to imagine that 100 years ago women didn't have the vote, yet they were going to a doctor's office to get masturbated."

In 19th-century Britain, the condition known as hysteria – which the vibrator was invented to treat – was not a source of embarrassment at all. Hysteria's symptoms included chronic anxiety, irritability and abdominal heaviness, and early medical explanations were inclined to blame some or other fault in the uterus. But in fact these women were suffering from straightforward sexual frustration – and by the mid-19th century the problem had reached epidemic proportions, said to afflict up to 75% of the female population. Yet because the very idea of female sexual arousal was proscribed in Victorian times, the condition was classed as non-sexual. It followed, therefore, that its cure would likewise be regarded as medical rather than sexual.

The only consistently effective remedy was a treatment that had been practised by physicians for centuries, consisting of a "pelvic massage" – performed manually, until the patient reached a "hysterical paroxysm", after which she appeared miraculously restored. The pelvic massage was a highly lucrative staple of many medical practices in 19th-century London, with repeat business all but guaranteed. There is no evidence of any doctor taking pleasure from its provision; on the contrary, according to medical journals, most complained that it was tedious, time-consuming and physically tiring. This being the Victorian age of invention, the solution was obvious: devise a labour-saving device that would get the job done quicker.

There were experiments in the mid-19th century with a wind-up vibrator, but it proved to be underpowered, with an unfortunate tendency to run down before finishing the job. A French pelvic douche appeared in the 1860s, which fired a jet of water at the clitoris and was claimed to induce paroxysm within four minutes; and by the mid-1870s, a steam-powered "Manipulator" had been invented, consisting of a table with a cut-out area for the patient's pelvis, to which a vibrating sphere was then applied. But both machines were complicated and cumbersome, and they were soon supplanted by the world's first ever electromechanical vibrator, complete with detachable vibratodes. Patented in the early 1880s by a London physician, Dr J Mortimer Granville, it predated the invention of the electric iron and the vacuum cleaner by a good decade.

At first, being powered by a generator the size of a fridge, the device was installed only in doctors' surgeries and operated by medics. But domestic electrification soon brought smaller, more portable consumer versions resembling hairdryers, and by the turn of the century more than 50 varieties were on sale. One manufacturer even offered a vibrator attachment for a home motor that could double up by driving a sewing machine.

For the next 20 or so years, the vibrator – or "massager", as it was known – enjoyed highly respectable popularity, advertised alongside other innocuous domestic appliances in the genteel pages of magazines such as Woman's Home Companion, beneath slogans describing them as "Such delightful companions", and promising, "All the pleasure of youth... will throb within you". In 1909, Good Housekeeping published a "tried and tested" review of different models, while an advert in a 1906 issue of Woman's Own assured readers, "It can be applied more rapidly, uniformly and deeply than by hand and for as long a period as may be desired."

Did women really not know what they were buying? Despite the lack of evidence to suggest otherwise, it seems unlikely – and the manufacturers surely knew what they were selling. Some of the language of early 20th-century ads is heavy with unmistakable innuendo, one boasting of its wares' "thrilling, invigorating, penetrating, revitalising vibrations", guaranteed to create an "irresistible desire" in a woman to own one.

Yet almost no one ever referred to the "hysterical paroxysm" as an orgasm – for how could it be? It was taken for granted in Victorian England that, in the absence of penetration, nothing sexual could possibly be taking place. A discreet veil of medical decorum survived until the late 1920s, when the appearance of vibrators in early porn films rendered the pretence untenable, and the vibrator promptly disappeared from polite public view.

It re-emerged in the 1960s as a rather daring sex toy, but according to Shere Hite's famous survey of sexual behaviour in the 1970s, only 1% of women had ever used one. This was perhaps unsurprising, given that most vibrators by then were modelled on a very male notion of what a woman would want – a supersized phallus – replicating, in other words, the very anatomy whose shortcomings had precipitated the invention in the first place.

But in the past 15 years the vibrator has undergone something of a renaissance. It began with the invention of the Rampant Rabbit in the mid-1990s – a model that features a clitoral stimulator, and was popularised by its appearance in a Sex And The City storyline in 1998. The advent of internet shopping also helped; when Ann Summers went online in 1999, the store sold one million Rabbits in 12 months, and annual sales in the UK continue to outstrip those of washing machines and tumble driers combined. Inspired by its success, other manufacturers have designed models that pay closer attention to the female anatomy than the male. Not so surprisingly, a 2009 US academic survey found that just over 50% of women had used a vibrator.

In the past 15 years, the vibrator has undergone something of a renaissance. It began with the invention of the Rampant Rabbit in the mid-1990s. Photograph: PR shot

Much of what we now know about the history of the vibrator comes from a small academic book by an American historian, wonderfully entitled The Technology Of Orgasm. Published in 1999, it is striking, and rather telling, that despite being such an interesting tale, no account of the vibrator's history had existed until then, either in academia or popular culture. Historian Rachel P Maines' book helped to inspire both a Broadway drama, The Vibrator Play, which won three Tony nominations, and a film, Hysteria. But the obstacles encountered by both Maines and the makers of Hysteria would appear to suggest that enthusiasm for the story is far from universal.

All three of the producers, one of the two writers and the director are women, and joke that this is no coincidence. "I think it makes it more acceptable to have women telling the story," suggests one of the producers, Judy Cairo. "If you had a guy telling the story of the vibrator, it might feel exploitative. Whereas women doing it says this will be fun." Auditions for some of the female parts were certainly a laugh, they say. "It was When Harry Met Sally, over and over again."

It is now more than a decade since producer Tracey Becker first came up with the idea. "We kept trying to get funding, but a lot of financiers were afraid of it. The companies are run by men, and every time a woman read the script they were interested, but then they'd bump it up and it got to the men's desks, and the men would be afraid of it."

"Pretty much any woman who read it got it," agrees Tanya Wexler, the director. "And all the gay men got it. But maybe a lot of straight guys thought it was going to be pornier than it was; maybe there was a little bit of fear of looking sleazy." Yet, as the writer, Jonah Lisa Dyer, points out, "It's really a film about female empowerment."

Interestingly, Maines encountered similar unease – if not outright hostility – while writing her book. Very soon after the publication of her first article on the vibrator, in a library newsletter, her position as a New York university assistant professor was terminated. "It was feared," she discovered, "that alumni would stop giving money to the school if it was discovered that a member of its faculty was doing research on vibrators." A further article, published in an academic journal called Technology And Society, almost caused the publication to be closed down.

Very soon, Maines grasped "what I should have realised all along: that some people, most of them male, take my findings personally and resent them as implied criticism".

If the story of the vibrator tells us anything, she argues, it is that men have been determined for millennia to deny the most obvious truth about women's sexual requirements. Explanations for this collective denial have ranged from profound fear of female sexuality to sheer laziness. Either way, Maines says, "The constant from Hippocrates to Freud – despite breathtaking changes in nearly every other area of medical thought – is that women who do not reach orgasm by penetration alone are sick or defective." Western society has steadfastly preferred to pathologise around 75% of the female population as frigid, hysterical or, as the Victorians liked to say, "out of sorts", than acknowledge the inconvenient truth that coitus might not be entirely satisfying to women.

"In effect," as Maines puts it, "doctors inherited the job of producing orgasm in women because it was a job nobody else wanted." The vibrator inherited the job when they got tired of it, too.

• Hysteria is released on 21 September