The Telegraph picked a particularly shrewd moment to pry into disabled people's care plans, seeing as we are currently are the disproportionate target of a cost-cutting campaign that has started with the poorest and most disadvantaged people in society. The article, reproduced later in the Mail and Express, claims to have found evidence of "taxpayers' money" spent on sex services for disabled people.

This is not a story about "taxpayers' money" – most disabled people who have local authority-funded care plans are only allowed to spend these on basic services such as help with washing and dressing. What it is really about is moral outrage over an isolated case, which is also a smokescreen for much more disturbing attitudes towards disabled people's lives.

The debate around disability, sex and using prostitutes is nothing new. The initial rhetoric is always the same: "Those poor disabled people who can't get sex. It's tragic, really." From there, opinion usually splits between liberals who are happy for disabled people to frequent brothels, feminists who reluctantly disagree, and a few people who admit to finding the idea of disabled people having sex a bit icky. Everyone agrees, uncritically, that being disabled means sex is scarce.

Those assumptions are loaded with prejudice and myth, and some debunking would be in order:

• Assumption 1: It's tragic but inevitable that disabled people can't find sexual partners.

In the Observer's Sex uncovered poll, 70% of respondents said they would not consider having sex with a person with a "physical disability". It would have been interesting to compare responses for those with other types of impairments, but the conclusion remains. In what is widely considered a diverse and inclusive society, we are still considered too disgusting, or pathetic perhaps, to sleep with. There are all manner of factors that make it more difficult for disabled people to meet prospective sexual partners, not least poor physical access to social venues – but the biggest obstacle is other people's attitudes.

This has as much to do with ignorance as it does with our culture's obsession with perfect bodies. Disabled people get used to being asked whether we "can (or do) have sex". The same people are probably surprised that wheelchair users can drive, and that deaf people go to nightclubs. And with the distaste for the idea of disability and sex comes the notion that if disabled people are sexual at all, they must be heterosexual. So that explains why I can't get into most of the bars on the London gay scene, then.

• Assumption 2: Paying for sex is therefore a right for disabled men.

Many people who consider themselves to be liberal would like to believe that the sex industry is full of emancipated professional women and that disabled men harm no one by visiting them. In her book, The Equality Illusion, Kat Banyard exposes the disempowerment and exploitation of prostitution: 68% of sex workers develop post-traumatic stress disorder, for example.

If it's appropriate to educate non-disabled people about the realities of prostitution, excluding disabled people is patronising at best. And yet, I've heard feminists generally opposed to the sex industry saying "but what about disabled people who need sex?" They are not only buying into the myth of the undesired, undesirable disabled person, they are also denying us a voice when we are quite capable of speaking for ourselves. If sexism in society can be challenged, so can this prejudice.

It's also rarely explicitly said that this debate is about disabled men. There are few stories of disabled women using sex services – the concern seems to be about men's need for sex: the representation of a sex-starved man and desexualised disabled person in the same body. This is a confusing and unfair image of disabled men, who are just as capable of making respectful choices towards women as their non-disabled counterparts.

Otto Baxter, the subject of a recent BBC documentary whose mother wanted to pay for a prostitute for him, reportedly decided he would rather make his own independent choices around sex. While that may not be what every disabled person decides, given the way many people see us, he chose not to accept the myth that no one would ever be attracted to him.

Disabled people's sexuality is a social issue. Many of us are having sex. We choose both disabled and non-disabled partners. We come in straight, lesbian, bi, gay and trans varieties. We're at it – get over it. For for those disabled people who want to be at it but aren't, the reasons are endemic in our society. They can be seen in the patronising language used in media commentary and in the irresponsibility of discussing confidential care plans at all.

That "icky" response towards disabled people having sex is not about evolutionary impulses – a fashionable excuse by people who want to justify feeling this way. It's about ignorance. That same distaste is to blame for some real human rights violations, such as the right to marry and found a family: a recent case brought under the Mental Capacity Act is just one of many examples of disabled people who are forcibly prevented from having children. And this ignorance, not the personal intricacies of people's sex lives, is what we should be discussing.

Disabled people may need support, advice and even training in sex – but what we need most of all is equality in society. That not only means better physical access to all sorts of venues but also a change in social attitudes, including in the media. When we are seen as equal people, equally sexual people, we will be empowered to move on from the idea that we can only have sex by exploiting others.