Two months ago, American canvas weevil was found in the archives of the National Gallery. After ravaging the Metropolitan and Guggenheim collections in the US, it is now working its way through the gallery's storage rooms. Hundreds of old masters have been destroyed. The curators fear that much of the collection will go the same way.

The media has greeted this devastation with near-silence. A front-page article in the Guardian's review section dismissed expressions of love for the paintings being destroyed as "bourgeois escapism". Those seeking to arrest the spread of canvas weevil were compared to "anti-immigration demagogues [who] claim that foreigners will destroy a unique and distinctive British culture".

Inconceivable? You would hope so – and this story is, happily, fictitious. But the responses I've mentioned are real, when you swap art for nature.

Earlier this year, the former energy minister John Hayes described concerns about the rainforests of Malaysia and Indonesia – which, with their tigers, orangutans and thousands of unique species, are being destroyed to grow biofuels – as "bourgeois views". (Coming from a Tory MP, this was magnificent.) In the Guardian Review on Saturday, Steven Poole took up his call (Comfort of the wild, 6 July). Echoing Mao's denunciations of pre-revolutionary Chinese culture, he castigated those of us who write about our love of nature as bourgeois and snobbish, and suggested that our concerns about the spread of exotic invasive animals and plants are a form of crypto-fascism – "the green version of the English Defence League".

Exotic invasive species are a straightforward ecological problem, wearily familiar to anyone trying to protect biodiversity. Some introduced creatures – such as brown hare, little owl, field poppy, corncockle and pheasant's eye in Britain – do no harm to their new homes, and are cherished and defended by nature lovers. Others, such as cane toads, mink, rats, rhododendron, kudzu vine or tree-killing fungi, can quickly simplify a complex ecosystem, wiping out many of its endemic animals and plants. They have characteristics (for example, being omnivorous, light-excluding, toxic or inedible to any native carnivore or herbivore) that allow them to tear an ecosystem to shreds. These aren't cultural constructions. They are biological facts.

Comparing those who describe this process to racists is the intellectual equivalent of stating that evolution through natural selection is a coded attack on the welfare state, or that the first law of thermodynamics was hatched by green campaigners intent on conserving energy. It is to see the words but not to understand the science they describe. This fallacy – mistaking scientific findings for cultural concepts – was deliciously ripped apart by Alan Sokal's satirical paper Transgressing the Boundaries: towards a transformative hermeneutics of quantum gravity.

I see a love for the diversity and richness of nature as an aesthetic and cultural impulse identical to the love of art. It is a form of culture as refined and intense as any other, yet those who profess it tend to be regarded as nerds, not connoisseurs (that's true snobbery for you). Poole and people like him position themselves among the philistines – those who see no value in the wonders with which others are enchanted.

Consider the issue of dry rot in historic buildings. It's a major problem. Anyone who dismissed the concern of conservators as a form of neofascism would be considered insane. Dry rot is an exotic invasive species, a fungus that, until we introduced it in shipments of timber, lived quietly on pine and yew trees in the Himalayas. Unchecked, it could destroy much of our cultural heritage. What's the difference?

Why is this Red Guard philistinism directed at those whose hearts are broken by the heedless destruction of the natural world, by people who wouldn't dream of trivialising the heedless destruction of the Bamiyan Buddhas, or the demolition last month of a 4,000-year-old pyramid at El Paraiso in Peru?

I think there may be three reasons. The first is ignorance. A complete absence of cultural understanding would be career death in the media. A complete absence of scientific understanding is no impediment at all, as almost all media outlets are run and dominated by humanities graduates. I think, among some commentators, there's also a sense that concern for the living planet is a check on human progress, an affront to the view of humanity as deus invictus, the weightless god, floating above the grubby realities of life on earth.

But most important, perhaps, is an unconscious absorption of the demands of money. Unlike most art, the wonders of nature often stand in the way of attempts to extract resources or to build airports or shopping centres. Corporate attacks on people who love and seek to defend the natural world have seeped into every pore. Culturally hegemonic, the developers' view finds expression in the most unlikely places.

So those of us whose love of the natural world is a source of constant joy and constant despair, who wish to immerse ourselves in nature as others immerse themselves in art, who try to defend the marvels that enthrall us, find ourselves labelled – from the Mail to the Guardian – as romantics, escapists and fascists. That, I suppose, is the price of confronting the power of money.

Twitter: @georgemonbiot