Is there anything good to be said about male sexuality? That might seem a daft question. Apparently it brings a lot of pleasure and excitement to the lives of men and women alike, it's inspired some of the greatest art, music and literature through the ages and has played a fairly substantial role in sustaining our species and populating the planet. Nonetheless you'll need to search very, very hard to find any positive appraisal of male heterosexuality.

Since the era of the permissive society and the mainstreaming of modern feminism, western society has gone a long way towards liberating women's sexuality. Younger women have, to an unprecedented extent, been encouraged to believe they can be as sexual as they like and to experience and express their desires as they wish. Even the age-old proscriptions on female promiscuity have been largely broken down, exemplified by the glorious flowering of the SlutWalk movement.

Simultaneously, and perhaps not coincidentally, male sexuality has been increasingly seen as a problem. You can hear it in the gentle, dismissive mockery that says men are simple creatures who "only want one thing" or, at the extreme, outright vilification. The male gaze threatens, male desire is aggressive. Our primal instincts are pathologised with the jargon of gender studies. Righteous and necessary efforts to reduce sexual crimes have had the unwelcome effect of teaching generations of men that our sexuality can be dangerous and frightening.

Don't believe me? Look back at the Bailey review into the early sexualisation of children, and the surrounding media hoo-ha. Leaving aside any concerns about the veracity and accuracy of the report itself (and I have plenty myself) it is striking that acres of print were devoted to the impacts of these social trends on girls, their self-esteem and body image; their developing sexuality; their safety and security. Barely a word was spoken about boys, beyond fears that they are being turned into beasts.

Again and again the message came out: girls have problems. Boys are problems. And yet does anyone doubt that there should be concerns about how easy access to porn impacts upon boys' sexual development, their self-esteem, their body image or performance anxieties? It's not as if young men bask in perfect mental health and happiness – young men commit suicide at nearly four times the rate of young women, and sex and relationships rank high on their list of concerns.

At the other end of the age range, sexually active older women are now widely eroticised (albeit often with a rather misogynistic undertone) as "cougars" or (forgive me) "Milfs" while their male equivalents are disparaged as dirty old men. Observer columnist Viv Groskop recently went further, opining about any older man who has sex outside marriage, even the mild-mannered old janitor John Major, saying "Unfortunately it's not against the law to be an old lecher. Maybe it should be. Or at the very least you shouldn't be rewarded with the highest office in the land."

Perhaps the greatest concern for men and women alike should be the way male sexuality and sexual expressiveness balances on a narrow tightrope of acceptability. One step off the wire and you tumble into the realm of perversion. As feminist blogger Clarisse Thorn noted last year, any man who hits on a woman and gets it wrong risks being branded a "creep" – sometimes deservedly so, of course, but often for no greater sin than being insufficiently attractive or socially skilled, or having misread a perceived signal of invitation. I've never heard of a woman being stigmatised or disparaged for expressing an attraction to big men, rough men, geeky men or whatever. A man who expresses similar desires for women who don't conform to standard norms of beauty is a perv, a fetishist, a weirdo.

All of these prejudices are rehearsed and reiterated by men and women alike, they reside in the intangible web of social norms, conventions and culture, but they can and must be challenged and changed. If we can begin to openly and joyously celebrate the positives to male sexuality, it might become easier for men to be happy and confident sexual partners, and in turn become better lovers, and sometimes better people.

Male sexuality is no less diverse, complex and wonderful than women's or, for that matter, no more base, coarse and animalistic. Sure, most men might be slightly more likely to let our gaze linger on eye-catching curves, and slightly less likely to giggle about our lovers' proclivities with our friends, but in the grand picture women and men are surprisingly similar, in this respect as in so many others. Women have been entirely justified in asking that we blokes respect their rights, autonomy and wishes, that we respect them as sexual beings. It shouldn't be too much to ask for a little of the same in return.