With the world as it now presents itself, there is something perverse, and probably dysfunctional, about a person who stays in the same house for 40 years. What about the expanding family syndrome, the school-lottery migration, the property portfolio neurosis? Have you no imagination? Don't you feel that incremental slide, familiar to anyone dull enough to have slept under one roof for more than six months, into life-change anxiety? We know the symptoms so well. Sirens, drills, helicopters sweeping low with blades set for maximum acoustic impact: crack up. Crisis.

Urban regeneration is a perpetual, self-renewing process; as soon as a five-storey development is nearing completion, it is revised into a 20-storey monolith. The catalogue of annoyances, sending inner-city dwellers away on their travels, is universal. Road rage before you fight your way out of Tesco's car park. Caught on camera doing 37mph down a deserted Mile End Road at 6am: with the generous offer of a £90 speed awareness course instead of points on your licence. The thief who hoists the laptop while you are waiting for the carpenter to put the shattered door back on its hinges. Any reasonable Hackney incomer would say that there were enough final straws, in this demented territory, to stack a meadow of Monet haystacks.

But, like the generations of chewing gum in the ridges of my worn-down trainers, I stuck. The house on Albion Drive in newly fashionable E8 (written up in the colour magazines just in time for the crash) was under the shadow of a compulsory purchase order when we bought it in 1968. Property ownership back then was unthinkable; my career was a procession of black bags and rented Streatham rooms with Polish landladies. I crept up the Northern Line from south London to Belsize Park and Hampstead. Shared bathrooms. Question marks of curly hair in pink soap. Then I lurched deliriously off-piste for a weekend in Hackney that stretched into decades of affectionate, exasperated sleepwalking.

Download the new Independent Premium app Sharing the full story, not just the headlines

The friends from the communal house where we first lodged came into a windfall inheritance and were plotting to take possession of a small piece of London. Should we follow their inspiration? The first house I looked at was the right one, cast-iron stairs descending from a first-floor kitchen into a narrow strip of garden, waist-high grass. There was a dim recognition, at once, that this was the place to be; for that one-off experiment, a life.

Taking possession of a Hackney house was an uneasy karma. We accepted the first two or three burglaries as a deserved toll, the price of our invasion. Even the police who came round, reluctantly, to take the details, implied that we had brought it on ourselves by parading our difference, unsmart casuals with too much hair and too many books. Their visit, with the old trick of asking to use the loo as a way of sussing out the premises for unlicensed herbal remedies, ended on a note of sympathy: "Terrible, terrible, absolutely shocking to see a room left in that state." They didn't realise that nothing upstairs had been touched, this was how I worked, up to the elbows in papers, paints, maps, stones.

And as for the lavatory, it was outside, rustic, separate. A nail driven into the brickwork alongside the door supported a tin bath. Hackney was still in its post-war austerity limbo. Prefabs, crumbling community centres, public slipper baths where we wallowed in giant tubs but had no control of the taps. The council was prepared to chip in with half the cost of installing interior plumbing and we accepted the gesture.

Recent immigrants, so they reported when I interviewed them for my latest book, Hackney, That Rose-Red Empire, were offered spacious villas in the John Betjeman-approved Albion Square. The square was constructed, on the estate of Sir William Middleton, between 1844 and 1849. Individual builders worked on groups of houses under the supervision of the architect George Pownall. It was a speculation, a punt, a way of exploiting the spirit of the times: railways, canals, the finishing industries that had mushroomed around Shoreditch. City clerks, police inspectors, upwardly mobile tradesfolk could aspire to a style of life that was tactfully self-regarding and inward – but folded around a garden, an oasis of London plane trees that was open to all. This shady patch was restored by the Metropolitan Gardens Association in 1899 and provided with a public drinking fountain.

The newcomers, economic migrants from Cyprus, were unpersuaded. By the late Sixties, many of the grand houses of Albion Square were uncared for, on the market at around £5,000. The migrants demanded a nice new flat in one of the tower blocks. It was rumoured that the senior Kray brother, Charlie, owned the house on the corner, but he was unavailable for comment, out of town for years at a stretch. The coming-home party was held in the Albion pub, now an attractively customised private house, the signboard retained as an interesting feature. Most of the local pubs, by which social and political allegiances were confirmed, have been demolished. Of late the demographic has shifted dramatically. Recent incomers to Albion Square, where prices touch £1m, include a High Court judge.

***

When we arrived in 1968, the balconies of the flats at the back were sun traps where old white folk basked. They kept a close eye on our bodged attempts to install a bathroom, to break up the concrete path and demolish the extension with its asbestos-insulated roof. We were soon on visiting terms. They rang my wife to report any suspicious behaviour on my part, underground films shot with a rabble of students, suspect images projected on the trees at night.

Right from the start, I was responding to qualities of place that I barely recognised and didn't understand. In the 8mm diary film we launched at this moment, my neighbour Mr Duck is painstakingly grooming his tight, 30-yard strip; the immaculate beds, the weed-free path. His shoes are polished, waistcoat buttons gleam, his shirt is crisp and white. The widowed Mrs Fleming, in severe housecoat on the other side, clipped her flowers to a standard which

I perceived as a rebuke to our food-for-free chaos of beans, tomatoes, rhubarb, raspberries. We enjoyed conversations over the garden wall, but weren't invited indoors.

These were the first houses built on this ground. The householders of the Sixties, established, proud of what they had, respected a legacy of small farms and market gardens. The lovingly tended gardens were like a jigsaw of emerald carpets rescued and restored from the age of agricultural enclosures. But these old Hackney folk were the end of the line, hanging on to something that was no longer there: a community of individualists keeping up self-imposed standards as budgie fanciers, constructors of sheds. Their homes lay under the shadow of compulsory purchase orders. Some chose to carry on, staying where they had always been, until they keeled over, broom in hand, or died without fuss in their familiar beds.

The younger families, kids and grandparents under one roof, decided on a second evacuation: to Ongar, Loughton, the fringes of Epping Forest. They were nervous of the energies of the new Hackney, new patterns of crime; a climate of aggressive interventionism from councillors rabid with half-digested social theory. The noble dinosaurs of Mesozoic Marxism still took their holidays in East Germany and waited for the day when tower-block estates alongside the River Lea would be worthy of Erich Honecker's utopian regime. Young-blood careerists imposed post-architectural fantasies on our guinea-pig borough, driving out the last remnant of long-settled, white working-class property owners. And opening the door to tribes of warring leftists, squatters, community artists, occasional labourers, improvising musicians and the vocationally disgruntled. Most would drift onwards and outwards. Some would still be there, with all their campaign scars, when the money metaphor collapsed, the markets tumbled and fresh activists battled over threatened theatres, occupied cafés, padlocked swimming pools.

***

The terraced house we contracted to buy cost less than £3,000. Or: the fee I earned for my part in a TV documentary on Allen Ginsberg made for WDR in Cologne. Cash in a carrier bag, collected from a hotel on Park Lane, I travelled east across town to secure the Albion Drive property where we would spend the next 40 years. The family making the sale admitted that they could have negotiated a higher offer, but were keen to have a young white couple take over their home. Research, years later, showed that the Aglands had been here since the war. And that their house and .047-acre garden were presented with a compulsory purchase order in 1946. A crack, vibration from bomb damage, ran down one wall. The abbreviated terrace of Albion Drive came through that purge, to sit between reefs of social housing, a tolerated memento to another age.

We inherited a more recent sentence of doom. My expectation was for five years of Hackney life, walks, expeditions, the casual engagement I had enjoyed in Brixton and Belsize Park – and then away, purchase price returned, as the bulldozers trundled in. Profit was not part of the calculation. The destructive morality of using your home, that basic human unit, as a form of gambling didn't arrive until the late Seventies, the dawn of the Thatcher era. And with that restless philosophy, greed and neurosis took over: move in, move on, trade up. Debt became the measure of maturity. All the accidental benefits of an exhausted, punch-drunk regime – unheated lidos, Edwardian bathhouses with flaking plaster, overwhelmed hospitals, sprawling street markets – were challenged, rationalised, decommissioned.

By that fortunate accident of acquiring a base, a freelance anti-career became possible. All my reserves had gone into buying this house, so I was in the strange position of being the owner of a condemned property; a person without funds or prospects. But council taxes and utility bills were modest, we lived a loosely communal life, folk came and went (some offered a few pounds in rent). A friend, extracting himself from a failed marriage, arrived for the weekend and was still there two years later. We rarely saw him. He left early, to work in an office on the other side of town. And he returned late. Now and then, when I was making my sandwiches before cycling off to my job as a Limehouse gardener, I would bump into an air hostess repairing her theatrical make-up.

Freedom from a mortgage, though I didn't realise it, was crucial. If I could save anything from my casual labouring jobs, I put it towards publishing another small-press booklet. Between 1970 and 1979, we got out 18 titles. Accounts were settled, one way or another, to the end of the decade, when costs suddenly leapt and subscribers vanished. The independent poetry world committed voluntary suicide rather than struggle on through the political darkness. By now, the compulsory purchase order on our house had been revoked and Albion Drive was rebranded as a conservation area. It wasn't long before leaflets arrived ordering us to reinstate a set of Victorian railings, with arrowheads and bulbous finials. Traces of the original railings were visible in shallow concrete like stumps in the mouth of a man who chewed stones. The iron, so we were told, had been sacrificed in the war, melted down to make shells or tanks.

The Seventies were pinched. I loaded and unloaded containers of sheep casings alongside the railway yards in Stratford, in sheds that are now part of the Olympic Park. There were random power cuts. I carried home broken chunks of wax to make candles. When officials came to the door asking for licences of various kinds, I was over the back wall and away to the post office that served the flats. When the post office closed, many of the local businesses went with it: bread and cakes, butcher, fishmonger, two fruit and veg shops. Only the betting shop thrived, taxing the eternal optimism of the economic underclass.

***

It was a memorable day when a rag-trade patron, a true son of Hackney, arrived on my doorstep in his royal-blue Rolls-Royce Corniche, to offer employment: as a cultural dowser, editor, New Age copywriter for the bookshop he was opening in Regent Street. Jeff Kwintner had found one of our Albion Village Press books, the documentary account of my filming with Allen Ginsberg, in a shop in West Hampstead. He decided to drive straight back to his old manor to put me on the payroll. Now, we could afford to buy a £15 baby buggy, a Maclaren from John Lewis. Our daughter, who was pushed down Queensbridge Road and Brick Lane in her new chariot to meet me at the gates of Trumans Brewery, where I worked as a labourer in the ullage cellars, was joined by a son, born in this house.

With the arrival of children, for ourselves and the friends who had come here with us, there were decisions to be taken. One group decided that now, the mid-Seventies, was the moment to move to the country, taking shares in a large communal property in Hampshire: with expectations of keeping the odd cow, making yogurt and cheese, planting kitchen gardens. Would we join them? I didn't think we could afford it. What work could I get, being pretty much unemployable and running out of the puff for manual labour? The property our friends bought in Albion Square, for less than £6,000, was sold, a reasonable price for the time, for £15,000. To the people who had printed our small press books.

The realisation also hit me, though I wasn't stupid enough to spell it out, that I was addicted to London, to Hackney. Here was my raw material, a job for life, picking at a mythology of place: subterranean conspiracies, lost writers, the action in street markets. Out of the blue, I inherited a collection of books from an old lady in Wales. I took a stall in Camden Passage in Islington. And I began my second education, my doctorate in survivalism. By standing in the same place, every Thursday, from 7am to 5pm, the library of the world came to me: people who were books, strange men dedicated to securing a single item (it would be death to find it). I experienced the covert hierarchies of the book trade, from the gutters and sacks of Sunday morning Cheshire Street to Covent Garden, Club Row to Savile Row. Weeks vanished out on the road, combing junkpits, chasing rumours. Petrol station coffee. One dealer's spectral paws were ingrained with black boot polish from the hours he spent buffing up tired Edwardian cloth covers.

Spending one day a week at home, preparing and pricing stock, meant that I barely remembered that I lived in Hackney. The house didn't fill with books, it was made from them; columns supported the walls, insulated the loft. Californian dealers on their summer migration clambered over our furniture to sift the treasures, filling boxes to be shipped out. Once, a pound coin rolled under the bed. I've never forgotten the expression on the face of the dealer's wife when I lifted the corner – and this immaculate, groomed and brittle woman was exposed to... London. Dustballs, historic newspapers, odd shoes, Biro caps, rubber bands. Fossilised vermin.

Having some sort of income meant that we were obliged to go into debt: this was the Eighties. With three growing children (our youngest daughter was born in 1980), and the ever-expanding stock of books (otherwise known as mistakes), it was a decision I couldn't deny: take a mortgage. To open up the loft as an additional store, to bring the kitchen downstairs, to put in another lavatory, to tweak an extra bedroom, cost 10 times as much as buying the house in 1968. Irish builders came and went, cheerfully knocking down walls, ripping out plumbing and then vanishing to Chelsea, until the weather broke. I had decided to take three months off to try and write my first novel. My father died. My mother began to drift. My wife went into hospital. And now this mortgage, like a hungry incubus, demanded constant attention: it fed on anxieties I had previously ignored. The mad logic of the period would have been to find another property, buy to let, in some converted asylum or luminous watch-dial factory.

Nobody holds their nerve long enough to stay for more than a few months in a house without combing the property listings, making weekend expeditions into the badlands of Thames Gateway. Toxic dumps that might become the next big thing. The freelance duck-and-dive years were over: I discovered multi-tasking. Publishing books, which often vanished as soon as they hit the shelves, was only a way of auditioning for scraps of radio work, journalistic crumbs, talks at outlying academic institutions (convenient for my hike around the M25). I carried on with the dealing, from one of those grim oxymorons conducted in Bloomsbury hotels: the book fair. Until I was given a lifetime ban for the crime of going home before the time specified on the card. Alternative career snatched away, there was no option but to speed up my rate of production: more books, more readings, longer explanations.

I hadn't really noticed, but the people around me changed. Everybody who turned up to do an interview, or to take a photograph against the bookshelf or down by the canal, lived in Stoke Newington. There were so many artists in the area that they were forced to attack the walls with their spray cans and stencils, taggers had to scramble up gasholders to find a free space. Broadway Market, a limbo of local cafés and barbers, was promoted, overnight, as the new Portobello Road: bistro to retro. Before the inevitable evictions and rent hike. London Fields, a dog patch with a closed lido, was revitalised for improvised picnics, weekend parties of wine bottles, Belgian lagers and excited chatter.

There was something about this area, about our mid-Victorian terrace. The neighbours, the ones who arrived after Mr Duck and Mrs Fleming, stayed put. An art historian and a Spanish lady whose children have grown up in Hackney. At different times, they both contemplated moving away, relocating to Canterbury or Barcelona. But it didn't happen. The place was used to them, and away from it, they would never be quite themselves. I understood. The urban package – modest attached house, small garden, proximity of markets and parks, just enough drama to keep things interesting – was an ideal. It worked. There was as much of a community as you needed. Enough anonymity too. The major developments, in all their destructive force, were close at hand, but didn't threaten to blow everything, on our immediate doorstep, into oblivion. We were goaded, hammered by bureaucratic interventions, but not driven out.

Researching my Hackney book, I found that a commercial traveller called August Wolf, a Prussian, had occupied this property in 1871, with his wife, Emma, a visiting family, and a domestic servant. Like me, he must have spent time on the road, before returning to the undervalued dispensation of domestic life. Houses set a tradition and those who move in come to terms with whatever the pattern of bricks and mortar, the layers of wallpaper, require. We are never more than an extension of the vessel that contains us. Rachel Whiteread, who lived on the western edge of Albion Square, understood this very well. Her House project constructed a monument to memory and texture, before it disappeared; setting up new narratives of absence and loss.

When the Irish builders uncovered a brick arch in a newly disclosed cellar, I let them drown it in concrete. It's my guess that this was part of one of the kilns with which the speculative developer fired bricks on his farmland. Local historians tell us we are living on ground once part of Lamb Farm, not far from the Pigwell brook. Rivers are culverted. Streets are demolished. Certain qualities, peculiar to place, continue, attracting those who are most susceptible to them.

A few years ago, children decamped, we found somewhere very appealing on the south coast. Our house went on the market and we received several offers within the week. That was when we understood, without discussing it, how impossible escape was. You can't leave the thing that you are, the house that has become your biography.

'Hackney, That Rose-Red Empire' by Iain Sinclair is published by Hamish Hamilton (£20). To order it for the special price of £18, with free post and packaging, call Independent Books Direct on 0870 079 8897 or visit www.independentbooksdirect.co.uk