In the past couple of days, the media have been full of stories about how Margaret Thatcher was secretly advised to abandon my home city of Liverpool to a fate of "managed decline" following the 1981 Toxteth riots. Official papers held at the National Archives, now open to the public after 30 years, show how senior ministers, led by Geoffrey Howe, urged her not to waste money on the "stony ground" of Merseyside following environment minister Michael Heseltine's request for £100m to regenerate the area.

In the end, Heseltine got his money, but it was hard to tell what difference the cash made. Though "the minister for Liverpool" visited once a week, the money did not seem to be spent with any imagination or effectiveness and in the 1980s in particular the city was caught in a crushing vice of economic decline and political conflict that no half-hearted intervention could reverse.

These days, the centre of Liverpool buzzes with energy. Around the Pier Head is the largest collection of museums and galleries outside London, with a gigantic mall and, at night, girls in tiny frocks, kept warm only by a 2cm layer of makeup and spray tan, teetering from fashionable bar to fashionable bar; you hear accents from across the country and languages from all over the world.

In the 80s, it was very different. As trains entered the tunnels that led into Lime Street station, a legend scrawled in paint on the wall by the side of the track greeted every traveller coming from the south. It read in big, black, letters: "Fuck Off All Cockneys", a message that remained untouched for 20 years. This angry legend appeared to convey a vivid sense of how the city felt itself besieged and persecuted by the metropolitan power located in the south. Or perhaps it was just aimed at visiting Chelsea supporters.

At the time, I remember, Heseltine's response to the riots seemed insufficient and faintly ludicrous. The main thrust of regeneration centred around not housing renewal or industrial intervention but a garden festival, which I duly visited. There were many dull, suburban flower displays and the inevitable yellow submarine. (Over the next three decades, Merseyside's once mighty shipbuilding industry seemed to contract until all it made was yellow submarines to be stuck outside airports and shopping centres.) My main memory of the day is that I met the TV scarecrow/simpleton Worzel Gummidge, played by former Doctor Who Jon Pertwee, who had been paid to cycle round in a bath chair looking jolly.

After the festival closed, the site was abandoned and remains so until this day. I drive past it on the way to my mother's house and it is a rather lovely spot right on the edge of the turbid river. Just as the prairie is coming back to the abandoned neighbourhoods of Detroit, so the festival site seems to be returning to the kind of broad-leaf English forest that has not been seen since the Industrial Revolution, an Arcadia where nymphs, shepherds and shelf-stackers from the Park Road Tesco's, Europe's largest urban superstore, frolic on Midsummer's Eve.

Over Christmas, I noticed some signs had sprung up on the site promising a forthcoming housing development, but I suspect that project has stalled. Apart from the festival, the only noticeable spending was on some shop fronts in riot-torn Granby Street; but when I filmed there a couple of years ago, that poor neighbourhood looked much worse than it had done in the 80s on the day after half of it had been burnt down.

Since the Thatcher story broke, Heseltine has been on the radio claiming that his actions somehow laid the groundwork that has led to the vibrant city centre of 2012. Maybe they did but my sense was that for more than 20 years there was only drift and decay, not helped by the dark days of the Militant-controlled city council.

On my trips home, I often got the sense that, especially in the decade after the riots, Liverpool became a kind of Petri dish for rival experiments in social policy. From the left, there was the Militant council and its pigheaded radicalism; from the right, there was savage deindustrialisation led by monetarist callousness.

The only visitors who seemed to travel to Liverpool back then were deluded idiots, political tourists who saw the city as the cradle of some kind of revolution. I recall going to the guard's van of a Lime Street-bound train to collect my bicycle and meeting an over-emotional, middle-class Trotskyist eschewing the bourgeois comfort of a seat, who referred to Liverpool as "the St Petersburg of Britain" and who was tremulous with excitement over being able to sit through some council finance subcommittee meetings.

In more recent times, housing in Liverpool has been hit hard by John Prescott's cretinous Pathfinder scheme. A while ago, Warren Bradley, the deposed Lib-Dem leader of the city council, admitted that their efforts connected with Pathfinder, along with the former Labour government and various regeneration charities, to compulsorily purchase and demolish thousands of 19th-century homes had left many communities "looking like war zones".

I took part in the campaign to try to save 371 repairable homes in the Edge Lane district, from a road-widening scheme connecting the city centre with the M62. Unfortunately, the new Labour council has pushed the plan through and as I drove up at Christmas I was greeted with rubble where a decent community had once been. The new homes that will replace the dignified Victorian houses are of a gruesomely banal design, which even the government's design adviser, the Commission for Architecture and the Built Environment, has criticised.

My experience is that today many things are better in Liverpool. On the first night of the 80s riots, my friend Ian walked home from the pub and noticed nothing different from any Saturday night. Now his Georgian neighbourhood is a tourist destination with smart restaurants and stylish boutique hotels where I stay when I am in town. It is just such a shame that there has been no holistic solution to the problems of post-industrialisation in the city, and so many neighbourhoods on the edge of the centre remain abandoned, tinned-up and waiting for regeneration cash that will now never arrive.