'More people everywhere. Dog walkers. Noise – the infernal machine. It's hard to hear grasshoppers or the wind in the trees. Little pockets of messy land. Untidy hedges. Derelict buildings – whatever happened to them?" Matthew Oates is listing the changes in our countryside over the past half century as we plummet through the leafy lanes of West Sussex towards his old school.

Fifty years ago this week, Oates was given a butterfly net for his 10th birthday. In the long grass by his school cricket pitch, he caught large whites and small skippers, and a passion for nature was born. Oates is still mad about butterflies, but he is also now an expert in the management of land to conserve insect life, and the country's leading authority on the elusive purple emperor. Every season, he has kept meticulous diaries about the changing weather and how plants and animals have fared.

He is an astute and unconventional observer of the profound transformation of the British countryside, and our changing relationship to it – the perfect person with whom to embark on a modest pilgrimage into the past, via the places in which he encountered wildlife as a child.

It is easy to focus on the losses. The conservation movement is gripped by a sense of crisis and doom, more so than ever this year. Concern for the environment has disappeared from the political agenda. Climate change, cutting carbon and renewable energy have some traction, but saving species from extinction, curbing development and protecting or restoring the countryside are ignored by decision-makers. Turtle doves and barn owls were this week revealed to be in crisis. The hen harrier, a magnificent bird of prey, has failed to breed in England this summer, and now stands on the brink of extinction. The State of Nature, a report published in May by a coalition of environmental charities, found that more than half our wildlife species are in decline. After futile attempts to work with the coalition, conservationists are now openly in despair: all mainstream parties, they know, are unwilling to see economic growth compromised by a concern for nature.

"We are heading for Armageddon in terms of nature. We are faced with a nightmare of no nature in large parts of southern England," says Mark Cocker, author of Birds and People, a monumental new account of the role birds play in human life. "The filigree of our lives, the things that make it fantastic – silver-washed fritillaries, nightingales, dunlin, water voles chomping away at the edge of the pool – are disappearing. We are faced with the most appalling loss."

Cocker's passion echoes a rollickingly fierce speech to the Welsh assembly earlier this summer by Iolo Williams, the RSPB officer turned Springwatch presenter, which is still being passed around and debated among naturalists. In it, Williams recalled his childhood by the Vyrnwy river, currently imperilled by proposals for pylons and infrastructure to service vast wind farms. The Mid Wales countryside may still appear glorious but, according to Williams, it is bereft: water voles, trout, curlew and cuckoos are gone; so, too, are insects and hay meadows (Wales has lost 99% of its hay meadows since the second world war). On the moors, alien conifers have been planted. "It's like going and looking at war graves. Every single tree is a death-knell, a nail in the coffin of that moor," said Williams. "We really are on the brink of disaster."

Oates and I began our pilgrimage at perhaps the best-protected environment of all: a British public school. His passion for nature was first sparked at Christ's Hospital school, founded for poor children and in 1902 relocated from London to 230 acres of countryside near Horsham. Its grounds look magnificent, and the cricket pitch where Oates caught his first butterfly is intact, of course. The outside world has pricked this leafy bubble, however. The sky, once full of house martins, is now virtually silent except for aeroplanes, and only one mud nest is visible in the eaves, where once there were hundreds.

Similarly, the tree creepers and holly blues that Oates remembers in profusion only appear in ones and twos. Instead, there is a ceaseless rumble of traffic on the A24. But this is not a straightforward picture of degradation. A buzzard cries as it floats overhead, and rabbits and deer pick through the woods, where there were none 50 years ago.

"The beauty of the place has actually increased. There's a lot less pointless lawn-mowing, it's a lot more bushy and it's still got this feeling of immense tranquillity," says Oates. While the school assures us there is a thriving ecology group and its 850 pupils are free to roam the grounds – enjoying the kind of access to wildlife untasted by most children – there is no old schoolmaster, as Oates once had, to run the Butterflies and Moths Club every Tuesday afternoon. Halfway through Oates's schooldays, the master retired and boys suddenly lost interest in butterflies. Children are unlikely to be inspired by nature if adults aren't.

As a boy, Oates would creep out of school and go butterflying in a nearby wood. We head there, down a surprisingly tranquil lane, and walk on to a flowery ride, full of silver-washed fritillaries. Another buzzard flies overhead. When Oates was a boy, conifers were planted by the Forestry Commission and non-native Norway spruce slowly strangled the life out of the wood. In the 1960s, Oates found rare species – the Duke of Burgundy, the pearl-bordered and the small pearl-bordered fritillary – but by the time he finished school, all three butterflies had become extinct here. The conifers had grown so tall and dense that no sunlight or wild flowers could survive below them, and the insects and birds vanished, too.

Over the last three decades, conservationists have at least won the argument against what Oates calls the "folly" of coniferisation. The Forestry Commission now removes conifers from ancient woodlands. In Oates's childhood wood, the conifers blew down in the great storm of 1987 and the wood was allowed to regenerate naturally. Twenty-six years on, it is full of tall native birch, ash and sallow.

"The scars of the 20th century have been wondrously healed," says Oates, smiling as we watch a purple emperor glide between the treetops. "It shows the power of nature when we stop interfering." It is not an unmanaged tangle, however: changes to woodland management are "a huge positive" over the last 50 years, says Oates, and in these woods the trees are thinned to enable wild flowers to thrive alongside butterflies. But the three species that became extinct in Oates' childhood have not returned, and are lost to most of the south-east.

Another thing is missing: children. "We were continuously outside," says Oates. "We played on farms. Now farms are a dangerous environment. Even farmers' children no longer play on the farms." A compelling brew of rational and irrational fears about traffic, stranger-danger and health and safety have stopped children growing up in nature. Like many naturalists, Cocker is concerned that wildlife is losing a battle with virtual worlds. "People are so cut off from their environment by iPads and smartphones and electronica," he says. "The loss of ecosystems is paralleled by the expansion of the digital ecosystem."

This squeeze on time spent in nature may be an unavoidable consequence of a society liberated from labouring on the land. Wildlife will invariably decline if we are oblivious to it, but Williams's speech identified a more specific culprit. He did not blame the farmers for pesticides or ploughing up hay meadows or foresters for planting conifers over the moors.

"They have just taken what money was available, they've used the grant system to do what they were encouraged to do," he says. "My anger, and it is an anger – it's a venom – is aimed at those grey, fat-salaried, spineless bureaucrats who sat by and watched all of this happen." More specifically, this means the contemporary conservationists who work for the likes of Natural Resources Wales (whose very name screams exploitation) and have failed to stop the destruction of our countryside.

Cocker disagrees. "That was one fantastic speech but he's fundamentally wrong. It's not 'them'. It's not some fatcat with a big pension – it's you, it's me, it's us. Iolo wants it simple – that he is the good guy and he can blame some faceless bureaucrats. But that is not the problem. The problem is we don't wish nature conservation to stand in the way of economic growth." And this boils down to individual behaviour. "I'm still eating meat, I'm still flying, I'm still doing all the things I shouldn't."

It is easy to despair, but Cocker insists he also has hope – if everyone becomes an activist. "You cannot wander through your life telling people of the wonder of butterflies," he tells me. "I've awakened from a nature-writing slumber to feel that if I don't take action I will go to the grave a lesser person, because we are fucked." Actions, he says, can be big and small. Gardens make up 1m acres of British land. If every garden was planted with 50% native species, that would be an enormous boost to insects and birds. Cocker is also getting political: a key player in New Networks for Nature, he hopes to persuade the chief executive of every environmental charity in the country to support a festival of nature.

However, conservationists are divided over the best way to save nature when Conservatives, Labour and, critically, ordinary people, are reluctant to put wildlife ahead of immediate financial self-interest. Green thinkers such as Mark Lynas argue that a "technofix" will save us; others, such as Tony Juniper, make a convincing case for the economic value of nature through "ecosystem services".

Putting a monetary value on a view, or a tree, may be the only language that modern capitalism understands, but many argue that basing conservation solely on appeals to pragmatic self-interest sells nature short. The save-nature-because-it-will-save-us (or make us rich) argument may rescue the honeybee, but it will never protect the nightingale or the water vole. And how will we save anything if we fear nature, or simply have no feeling for it?

Oates says the doom and gloom is because conservationists are overwhelmed with grief, loss and failure. "We've fought the wrong battle," he says. "We've fought a battle over land and the real conservation battle is in people's hearts and minds." That sounds good – but how could this battle be won? Rather than reducing the beauty of the countryside to a price set by an economist, I wonder if minds might be won by scientific identification of the health benefits of nature.

More research into how wildlife aids physical and mental wellbeing must be a good start – although it could, of course, show that a home in a Shanghai high-rise is as mentally and physically beneficial as life by a lake in Finland. "That's my nightmare," says Cocker, "but I'm pretty sure there will be evidence to prove we need the flowers and the birds."

Fostering greater heart for wildlife, however, can probably only come through time spent in nature. For most of us, the countryside is now a place of leisure, rather than work, and adrenalin sports are increasingly popular. Oates would prefer to see more walking, fishing, foraging or butterflying (with cameras rather than nets – Oates's old net is just for our pictures): activities that "get us still and quiet in nature as opposed to charging through it".

Children, he believes, need initial inspiration or encouragement, but are then better left alone. "They need time without adults. They've got to be able to explore, have fun, go and muck about," he says. While a forthcoming documentary, Project Wild Thing, brilliantly shows how parents need to be persuaded of the benefits of nature more than children do, Oates would also like to see more support for wildlife lovers at secondary school, where peer pressure dictates that nature is for geeks.

Oates's old school and woods look better today than 50 years ago, but the missing species show the long-lasting legacy of environmental destruction. The most uplifting moment of our pilgrimage came when we reached the chalk grassland of the North Downs. When Oates first visited Denbies hillside four decades ago, the meadows were overgrown and several butterfly species were on the brink of extinction. Now the grassland is grazed for conservation by the National Trust, and flowers and insects are thriving. A year after the worst summer on record for butterflies, thousands of chalkhill blues shimmer on the slopes.

"Yes there are horrible, irreversible losses, but nature is more dynamic than we think," he says. "The way butterflies have bounced back after last year's apocalyptic summer is just miraculous." He takes heart from Wordsworth: "What we have loved, others will love, and we may teach them how." No matter how urban we become, "this love of nature is innate in all of us", believes Oates. "It's just a matter of finding that catalyst. It is about realising the interconnectivity of everything and what our real needs actually are – and they are beauty, wonder, passion and love."

• This article was amended on 16 August 2013 because the original said hen harriers have failed to breed in Britain this summer. This has been corrected to say in England.