Two of us recently scrambled up a hill in West Lothian. Not realising there was an easier way just around the corner, we chose the steep north face, encountering on the lower slope sprightly nettle beds and forests of the tallest rosebay willow-herb I've ever seen. Each step through them set off clouds of floating seeds that stuck in our hair and clothing, and it was a relief to reach the clearer air of a grass valley where a fox's lair lay among a copse of birches. We made the final assault from this little valley's head, at first climbing at the diagonal and then taking the steepness head-on, our feet slipping in the scree.

The view from the top was magnificent. The elevation above the surrounding fields was little more than 300ft, but from here we could see the Ochils to the north and the Pentlands to the south, the strange cone of North Berwick Law away to the east and a shadowy mountain, possibly Ben Lomond, to the west. Edinburgh Castle and the Forth bridges stood in the middle distance. Closer to, among the yellow fields, aircraft rose silently from Edinburgh's airport, miniature trains slid to and from Fife and Glasgow, and lorries ran like little boxes along the M8 and M9. A mobile landscape that depended entirely on oil.

"Even the fields, too," my companion said, my companion said, pointing out that sowing, fertilising and cropping couldn't happen without oil. In fact, we stood on oil looking at oil, because our vantage point was a byproduct of it: Greendyke Bing, a spoil heap so vast that like the Great Wall of China it was once rumoured to be visible from space. It rose over almost a century, from the 1850s to the 1940s, as the repository of the waste created by distilling oil from shale in the dozens of coal-fired retorts surrounding its base.

The countryside to the west of Edinburgh specialised in this industry. Several years before the US sank its first productive oil well in 1859, Scotland was home to the world's first commercial oil refinery. In 1851, the chemist-cum-businessman James Young, later known as "Paraffin" Young, opened a works near Bathgate that produced lubricating oils and naphtha (for use as a cleaning solvent) from the shale found among West Lothian's coal deposits. Soon he developed technologies that produced paraffin for lights – Bathgate oil lit a quarter of London's lamps – and paraffin wax for candles. In the 1860s, when many of Young's patents expired, Scotland became gripped by oil mania as dozens of hastily established companies dug pits and erected retorts and refineries in a small stretch of countryside to the south of the Forth.

A way of life that had depended on farming and the patronage of great landowners – the Linlithgow and Rosebery dynasties – was brutally disturbed. During its first boom, the shale oil industry employed more than 30,000 people, many of them migrants from elsewhere in Britain. Existing villages grew at a rate bewildering to those who lived in them – Broxburn's population went from 660 to 5,898 in 30 years – while entirely new settlements of brick cottages, with perhaps a store or a working men's institute at their centre, appeared suddenly where no one had previously thought to live. On the eve of the first world war, West Lothian shale produced 27.5m barrels of crude oil, which was roughly 2% of then world production, and amounts now to 5% of what will flow this year from the UK sector of the North Sea.

By then it was producing petrol as well as its traditional paraffin and wax, but it could never compete with the naturally liquid oil that began to be shipped from Persia and then elsewhere in the Middle East; distillation meant heating a retort to 704C, with 160,000 litres of water required per ton of oil shale for the steam that was needed to yield ammonia. Wartime shortages and the fear of unreliable supplies kept a shrinking industry alive into the second half of the last century, but the last shale mine closed in 1962, and then it was gone. The pitheads, the retorts, the refineries and the narrow-gauge electric railway that connected them: all vanished, leaving the spoil heaps, the bings, as the most visible evidence that industry had ever existed. And as the industry's process was very wasteful – the proportion in weight of burnt waste to oil was more than four to one – the bings were huge. For more than a century they have defined the West Lothian landscape by their size and colouring – red, unlike coal tips, because burnt shale oxidizes into that colour from its original blue-black.

Greendyke, the biggest of them, rises to an expansive plateau like Table Mountain (with Broxburn rather than Cape Town nestling below), with spurs branching out to foothills and gulleys marked by the tracks of off-road sport. A walk over the top reveals the remains of what were possibly air-shelters or gun emplacements, suggesting that by 1939 large parts of the bing were no longer operational. What has happened since shows the transforming power of biology. Again unlike a coal tip, the waste is alkaline not acidic; the tremendous heat of the retort has made it non-toxic. At the last count, 86 plant species flourish on the bing, including wormwood, the creeping buttercup and the common spotted orchid as well as birches, hawthorns and alders. Animal life, as well as foxes, runs to hares, red grouse and skylarks.

To someone like me, who remembers the dying days of the shale industry and always imagined the bings as a sterile wasteland, this is surprising; though not as surprising as the decision made in the 1990s to list two bings, Greendyke and Five Sisters, as industrial monuments protected in law against excavation and reshaping by road builders who want their red waste as hardcore. A report commissioned by the local authority described them (rather chauvinistically) as "a focus of community identity in a population whose common culture of mining is slowly being eradicated by families of non-West Lothian origin taking up residence in the many new housing developments in the county". The bings, therefore, had a cultural, recreational and educational as well as ecological importance. West Lothian Council features them prominently in its logo.

It would be foolish to use the example of the bings in an argument for letting loose all forms of extractive industry, including fracking. The 19th century plunged into hydrocarbon fuel without a care in the world, other than for dirty collars and smudged whites on the washing line. Still, it is interesting that massive despoliation can eventually be treasured aesthetically, for its contribution to a locality's landscape and character. At Greendyke's northern edge, we looked down on Niddry Castle, a 15th-century keep where Mary Queen of Scots once spent a night and now listed under the same legislation as the bings, though not the one only a few yards behind it, which is now being scalped for hardcore. It seems odd that Victorian society cared so little for something so old, allowing it to be nearly drowned in hot shale fresh from the retorts. But the question remains, given oil shale's former place in the Scottish economy, were they wrong?