In the last few years censorship, or indeed the cry thereof, has become something of a meme. Some well-meaning group of people, usually from a demographic that is still struggling to achieve full equality, will raise objections to the meaning of a piece of art or literature, sometimes – but notably not always – requesting its removal. And then the rightwing press will go ballistic. Crotchety letters will be written to newspapers, rent-a-gobs will start sharpening their poison pens, those who raised objections will be abused and ridiculed – and then, after many column inches, everyone will forget about it until the next censorship row comes along.

This is what happened in the case of Lola Olufemi, the Cambridge women’s officer who had the temerity to suggest a more diverse, “decolonised” English curriculum and as a result had her photograph placed on the front page under the headline “Student forces Cambridge to drop white authors” (the newspaper has since corrected and apologised, as well it should).

So often, people such as Olufemi, who was politely making a completely legitimate point about the canon, backed up by a large number of Cambridge academics, are portrayed as hysterical snowflakes who can’t bear differing opinions. And yet it is always the right wing who become pant-wettingly upset about having their views and interpretations challenged.

Manchester Art Gallery recently utilised the censorship meme to spectacular effect by temporarily removing John William Waterhouse’s 1896 painting Hylas and the Nymphs in order to prompt a discussion about painting choice in galleries. People duly and predictably went ballistic. Parallels were drawn with Nazi Germany (why choose the Nazis as a comparison? Why not the censorship of the feminist avant-garde?), letters were written to newspapers … you see where this is going.

Even before #MeToo, we were becoming a society in the process of reappraising its artistic output, and its artists

The painting row came hot on the heels of a row in New York over the display of Thérèse Dreaming (1938) by the Polish-French artist Balthasar Klossowski de Rola, which has an 11-year-old girl showing her knickers as its subject. It is no coincidence that both paintings depict what appear to be pre-pubescent girls. The #MeToo movement is in full swing, but even before that we were becoming a society in the process of reappraising its artistic output, not to mention its artists – whether that is a statue of a racist colonialist/national hero (delete according to your viewpoint), the films of Woody Allen and Roman Polanski, and, indeed, the men themselves, or novels, such as The Great Gatsby, that some students have said require trigger warnings.

Despite what we so often read, art is neither one thing nor the other. While one viewer might look at Hylas and the Nymphs and see the young girls in it as “mildly erotic”, as this newspaper’s art critic did, another viewer might remark a paedophilic tendency that is, actually, not at all mild or subtle. I might look at the almost-breasts of the young women – sorry, girls – in that painting and remember when my own looked like that, how young I was, how vulnerable, how books and trees and rollerskating promised infinitely more pleasure than the body of a powerful adult male.

Would I ban Hylas and the Nymphs? Only on the grounds of cheesiness (this is a joke, do not write me emails). No, of course, I wouldn’t ban it, just as I wouldn’t ban filthy old pervert Degas and his pre-teen ballerinas. But I would like to see it as part of an exhibition that interrogates why so much of our artistic energy as a society has been devoted to sexually objectifying young girls.

‘Would I ban Hylas and the Nymphs? No, of course not, just as I wouldn’t ban filthy old pervert Degas and his pre-teen ballerinas.’ Photograph: Hilaire-Germain-Edgar Degas/Glasgow Museums: Art Gallery & Museums, Kelvingrove

All of this brings to mind, inevitably, Lolita. In response to calls for censorship, Nabokov’s acolytes have become furiously dogmatic about this novel and what it can be said to be about. I recall reading a New Yorker essay several years ago in which a male novelist examined his irritation at a statement that was made about the novel by his college professor. “When you read Lolita, keep in mind that what you’re reading about is the systematic rape of a young girl,” the professor said.

The novelist did not like this, but he was curious enough about why to do some self-examination. He talked to the feminist Alexandra Brodsky, who said: “What a delight it must be to read a book full of graphic accounts of sexual violence and still have the book not be about sexual violence to you!” she said. “Why is the depersonalised, apolitical reading the one we should fight for?”

In other words, why is your reading (or indeed your way of seeing) the correct way? Your interpretation of a work of art, as for example, a white male, is profoundly influenced by who you are, and can in fact be said to be as political as any other. And yet: your viewing is apolitical, objective, fair, while mine, or, say, Olufemi’s, is hysterical, shrieking, biased; a misinterpretation. Why?

In Men Explain Lolita to Me, Rebecca Solnit responds wittily to the charge that, in identifying with Lolita, the young girl who is systematically raped over the course of the novel, Solnit has “wilfully” misunderstood Nabokov (there is such confidence in that statement, such conviction in the rightness of one’s interpretation). But art is not one thing nor the other. Am I allowed to think that Lolita is a brilliant psychological study into the mind of a paedophile, but also wonder whether Nabokov, for whom paedophilia was something of a preoccupation and who may have been abused himself, got off on writing it?

We are missing an important moment if we refuse to listen to one another about art that could be deemed dodgy, if we consider our own interpretations of that art to be the only right ones. I know that nuance is unfashionable, that it is unlikely to become a meme anytime soon, but it is also the site in which most art resides. It would be such a shame if we lost it.

• Rhiannon Lucy Cosslett is a Guardian columnist