When word got out that Stephen Fry, the tweedy, cuddly face of acceptable bourgeois homosexuality, had said something rather offensive about women, a collective gasp rang out across the nation. He couldn't have, surely? Not lovely Stephen Fry, for whom even the most homophobic grandmother reserves a soft spot; not Stephen Fry, the only man whom both the drugs-and-buggery counterculture and the brandy-and-bigotry establishment recognise as one of their own?

Unfortunately, everyone's favourite gay uncle really has proposed that women only ever have sex for money, or to manipulate a man into a relationship. Despite claiming to have been misquoted by Attitude magazine, Fry is on record in several other interviews opining that women don't really like sex – for if they did, they would "go to Hampstead Heath and meet strangers to shag behind a bush."

Fry's words have upset many of his fans, and for good reason. The writer and broadcaster has a special genius as a chronicler of adolescence, of the indignities of lust and romantic adventure. For many of his followers, this wry empathy with an almost universal human imbecility in the face of groinal temptation is what draws us to his work.

Reading Stephen Fry's books was a huge comfort in my lonely, horny teenage years. As such, it is doubly hurtful that Fry now informs me that, as a woman, I lack the necessary equipment to understand sexual desire. As he elucidated in another interview, "you don't get … what it's like to have one of these in your bloody trousers!"

Many have queried whether Fry, a man with limited experience of a lady's intimate regions, has the right to pronounce on female erotic preference. Feminists across the internet have defended themselves against the wearisome patriarchal charge of frigidity.

The more profound question, however, is why Fry feels that females inhabit a world so alien to his own. There are, after all, significant points of similarity between the erotic experiences of gay men and straight women, not all of them anatomical. Both men and women who grow up desiring men understand what it's like to have one's desires frowned upon. Both men and women who grow up desiring men know what it's like to fear violence if we express our sexuality, especially if we're brazen enough to walk alone on the streets at night.

In this context, Fry's spectacular ability to entirely dismiss the culture of shame, sexual threat and social stigma associated with female sexuality seems offensively myopic. The national treasure's utter indifference to female experience is particularly chilling because he embodies a certain strand of cosy, unthreatening, upper-middle-class homosexuality within whose ranks misogyny routinely goes unchallenged.

Instead of solidarity in the face of a heteronormative patriarchy that oppresses all of us, there remains a chasm of suspicion and misunderstanding that obstructs genuine solidarity between women and gay men. Fry's words are a perfect expression of that process of mutual incomprehension, a process whereby our culture has become so alienated from its own sexuality that erotic impulses can never be a point of community, only of difference.

For example, despite the bland saturation of our aesthetic environment with images of "sexy" ladies, female sexuality is still a mystery to many men, gay and straight. A lot of men find it hard to grasp the many social reasons for female sexual reticence, and harder still to imagine that a woman's desires could have any similarity to their own.

Fry's belief in the rigidity of sexual difference overlooks centuries of sexist and homophobic oppression. There are many reasons why women don't tend to go looking for a bonk in a park – not only because a culture of shame and violence prevents many of us from seeking sexual enjoyment, but because cottaging and surreptitious alfresco sex first became popular among gay men when homosexual intercourse was still illegal.

The uncomfortable truth is that gay or straight, male or female, we all have kinky thoughts. Nearly everyone, whatever their particular proclivities, is liable to go a bit funny when presented with the prospect of a rummage in somebody else's pants. Fusty bourgeois refusal to accept that most people are simply gagging for it most of the time, including women and queers, remains at the root of most sexism and homophobia.

