To say goodbye to Istanbul after 28 years of living in the city, I take one of my favourite walks, from Pera’s Tünel square, wandering down the hill through Galata and then along the Golden Horn sea inlet. For me, this jumble of shops, alleyways and quaysides best conjures up Istanbul’s heady mix of peoples and history: Byzantium, the Ottoman Empire, republican Turkey, and a global megacity on the make.

I head off from my much-loved century-old apartment building, with my wife, my youngest daughter and, passing through like so many others, a childhood friend from South Africa. Some of these travellers are lucky, some not: outside the castellated gateway of my neighbour, Sweden’s two-century old consulate general, Syrians stand in line hoping for visas to Europe.

On the same street, tourists are busily clicking off photos of one of Istanbul’s red and white, Belgian-built electric street tram, restored to service in 1990 and now practically a symbol of the city. We pass by the arches of the Tünel short underground funicular railway, the first metro in continental Europe when it opened in 1875, and still the city’s only real metro in the city until 1999. We continue past the gates of the Galata dervish lodge and its hidden oasis of grand plane trees and mystical whirling.

The main street downhill, named for the 18th century dervish sheikh and poet Galip Dede, is well-cobbled, fitting its new role as a thoroughfare for the more adventurous tourist. The old lamp and car part shops have morphed into outlets for Chinese-made knick-knacks, thin, overpriced hamam bath towels and fresh-pressed pomegranate juice. But many of the old musical instrument shops are still doing well, selling everything from tambourines to white grand pianos and Istanbul’s contribution to the world’s musical vocabulary – the handmade brass cymbal.

Spare parts for almost anything

Half-way down the hill is the noble old stone cylinder of the Galata tower, part of the fortifications that protected late Byzantium’s Genoese colony, topped since the 1960s by a roof in the shape of a pointed cone. Here we part ways with the travellers who are juggling to fit both the tower and their own image into cellphone selfies, and head toward Perşembe Pazarı, a warren of shops that supply Anatolia with spare parts for almost anything.

There are countless places to pause: to trace parts of the pre-1453 Ottoman conquest fortification walls propping up lines of houses; duck into a diminutive early Ottoman courthouse turned aluminium depot; admire the curving street of grand Ottoman banks now becoming boutique hotels, fancy restaurants and cultural centres; and make another attempt to photograph satisfactorily my favourite survivor, a centuries-old stone and brick building colonised by electric motor repair workshops, whose angular first floor rooms jut out into the narrow street.

Toward the bottom of the hill come all manner of premises, from the unexpectedly cavernous to glassed-in cupboards in the wall barely bigger than the men inside them. They display their products proudly: sawblades as big as cartwheels, stacks of metal ingots, rubber seals of all sizes, mammoth industrial fans, and amazing varieties of nuts and bolts. Finally, just before reaching the shore of the Golden Horn sea inlet, a last jumble of ships chandlers sells great chains, yachtsman’s gadgets, and anchors that would take a crane to lift.

I lead our group to a place I’d spotted a few days before. Tables and chairs of a new generation of impromptu and entirely illegal restaurants have for a couple of weeks been spreading rapidly along the Golden Horn’s worn-out parks and quaysides of battered tour boats and fisherman’s skiffs. It looks popular and enchanting, and I want to have my last Istanbul dinner here.

The waterfront is already busy with an organised, illicit chaos. After a few minutes of bobbing and weaving, I see waving arms call us to a miraculous space at the water’s edge. A plaid-shirted maître d’ quickly has us balanced on rickety chairs, sipping Turkey’s aniseed-flavored raki, accompanied by slices of ripe honey melon and a slab of rich white cheese. There is no arguing with this director of operations; from a kitchen that looks better suited to a campsite than a restaurant, we are told that we are going to be served grilled sea bass.

An eternal rhythm of the ad hoc

The pop-up restaurant is the embodiment of alla turca, summing up a city that is so many contradictory things at once. Istanbul’s many beauties are often islands in a sea of concrete ugliness. The new can be piled in layers on top of the deeply ancient. A day cannot be planned and is therefore lived ad hoc; nevertheless, days pass in a rhythm that is apparently eternal. It is impossible for anything to be 100 per cent legal – the jumble of laws is too complicated for that – but a paternalistic, interconnected state discipline somehow keeps everything in harness. And personal touches of generous kindness are an essential oil that helps everyone to survive the increasingly tense pressures of a city with teeming millions of people.

This was the spirit that attracted me to the rough-and-ready city I stumbled upon 28 years ago. Thereafter I made many active choices to stay in Turkey. As a base, Istanbul is the heart of a wide and active geography, and as a home, the country offers everything any reasonable working person could want. The only missing part, perhaps, is something I never asked for: a legal sense of rooted, long-term, rightful belonging. I’d just keep extending my annual residence permit, making the most of life as a voluntary pot plant, always grateful for my place as a foreign guest.

Fasting is Cleansing

Sitting at our wobbly table on the bank of the Golden Horn, a bright star snugly fitting into the curve of a perfect Turkish crescent moon above our heads, I contemplate how much Istanbul has changed. In the 1980s the Golden Horn was filthy, devoid of marine life and reeked of sewage. In winter, the city’s mists and lignite coal furnace emissions used to mix and malinger as thick, sulphurous smog that choked me up as practically the sole, lunatic jogger in town.

Now, the Golden Horn is clear and bridges are lined with fisherman pulling up lines of silvery sardines. Natural gas has cleared the city’s lungs, and recreational walking and running are even fashionable. The city is at last planting some trees and grass, and fixing up the public spaces, even if some renovations are stripping the ancient patina from the city’s finest buildings and making them look like newly tiled public conveniences. Over the water, as for every holy Muslim month of Ramadan, bright lights strung out between the minarets of one of the Ottoman imperial mosques enjoin the population to obey the dawn-to-dusk fast. But now the call to the faithful has an almost health-fad ring to it: “Fasting is Cleansing”.

When I arrived in 1987, there were almost no buildings higher than ten stories and just two five-star hotels: a boxy, cookie-cutter 1950s Hilton and an angular, ugly glass-fronted Sheraton. Now the city has countless luxurious lodgings, from Bosporus-side boutique hotels to restored Ottoman mansions. One financial district has created a completely new skyline half-way up the European side of the Bosporus waterway, and a second one is rising on the Asian side of the city.

Three decades ago, there were no supermarkets at all, and restaurants served excellent but almost exclusively Turkish foods. Shopping was an art-form that required an encyclopaedic memory for what I might find where in the city; the guidebook of the day was like an 18th century Larousse, a jumbled, glossy, barely comprehensible volume that fell apart days after you bought it. Now, sleek new shopping malls reach up to the sky, crackling with brand names and glamorous eateries that attract customers from all over the compass, from Senegal and Riyadh and Paris and Omsk.

Turkey’s new manufacturing prowess has moved it away from the two main exports it had when I arrived: hazelnuts and dried figs. There’s no going back to the days when we huddled for an hour a week to see the outside world on the first live link to CNN on television – complete with simultaneous English broadcast via a radio channel. And I doubt I will ever again see the sour face of the lady at the Istanbul post office’s now defunct “small package service” as she dissected a gift of Swiss chocolates in search of contraband. Turkey feels more and more ‘normal’.

A self-conscious separateness

Yet, I cannot believe Turkey will give up on its self-conscious separateness. Even today, with its long Black Sea, Aegean and Mediterranean coastlines, the country can feel like an island, well-deserving its ancient name of Asia Minor. Endowed with water, sun and a vast, fertile hinterland, it has never had urgent needs from the outside world. As with geography, so with politics. Bruised by the imperial carve-up of its predecessor state, the Ottoman Empire, Turkey’s governments and elites still seem to feel safer keeping the world at bay with customs barriers, residence permit requirements and a sometimes prickly, go-it-alone foreign policy.

The Turkey I came to in the 1980s guarded one third of the Cold War iron curtain with the former Soviet Union and its satellites, cutting it off from its natural Balkan, Eurasian and Middle Eastern economic hinterlands. Visitors and tourists mostly seemed to be western Europeans, north Americans and Japanese. International sea traffic on the Bosporus was most often Soviet warships being checked out by the white launch we all called the “CIA boat”.

Now, the Bosporus is filled with oil tankers and freighters servicing the many states of the Black Sea. The tourist crowds have been joined by Russians, Chinese, Balkan peoples, Central Asians and above all Arabs, as befits Istanbul’s historic role as a crossroads of east and west.

The new Istanbul is also huge, perhaps double the size it was in 1987, and an official population of 14 million seems plausible. Hundreds of thousands of people walk each day along Istiklal Street, the boulevard outside my house in the centre of town. If London is the great metropolis at the western end of Europe, Istanbul is the continent’s eastern bookend, making everything in between seem provincial or suburban.

A different set of historical data

The changes are intellectual too. In the early years I learned to bite my tongue on many subjects – whether it was the Kurds (officially “mountain Turks” when I arrived), the role of the military (almost sacred, protected by all manner of laws), Turkishness (ditto), Armenians (victims of a “so-called” genocide), or the Greeks (“spoiled by Europe”). It wasn’t just that I didn’t want to run foul of the law, or offend my Turkish friends. I also came to realise that people educated in Turkey worked from a quite different set of historical data than what I had been taught.

Over the years Turkey found out how others saw it and I learned why Turkey felt as it did about its history and neighbours. Most people stopped viewing me automatically as the agent of a foreign power, and the constant litany of conspiracy theories abated. The Gezi park protests around Istanbul’s Taksim Square in 2013 marked the point where for the first time I felt on the same wavelength with politically active Istanbul, perhaps because this time the young middle class raised its voice instead of extremists from left or right. At last, slogans were actually funny rather than reflecting dark layers of despair, victimhood or oppression. The same social courage guards ballot boxes from tampering at election time, empowers independent reporters to defy sometimes massive government pressures and keeps parliamentary debate alive.

This robust spirit gives me confidence that Turkey will ultimately keep moving in a positive direction. The Justice and Development Party (AKP), which came to power in 2002, with its leader Recep Tayyip Erdoğan, now president, deserve some credit for that. Erdoğan and AKP won legitimate majority governments in three elections in a row. They improved the health care system, built fine new roads and public transport systems all over the country, ended routine torture and broke many taboos in search of the right settlement of Turkey’s Kurdish problem and interconnected rebel insurgency. Of course, AKP got on a train that was ready to leave the station, thanks to reforms that their opponents, including the military, had already put in place to get Turkey in line with international standards.

In fact, despite setbacks in the 1980s and 1990s, Turkey’s defining events were mostly stepping stones towards a future more deeply anchored in the West and European institutions. As a journalist, I reported on the 1987 application to join the European Union, the 1995 Customs Union with Europe, and the 2005 opening of negotiations for full EU membership. The 1999 acceptance of Turkey as a candidate to join the EU was the happiest national moment I can remember in the country, alongside, perhaps, the day a year later when Istanbul’s Galatasaray club won soccer’s UEFA cup.

Even today, despite the love-hate nature of Turkey’s ties with Europe, this relationship remains by far the strongest it has, powered by five million Turks in Europe (compared with just a few hundred thousand expatriates elsewhere), and the fact that European countries are responsible for half of Turkey’s foreign trade and three quarters of Turkey’s inward investment.

A lost sense of direction

For now, though, both Turkey and its European partners have lost their sense of direction. Nobody on either side expects Turkey to join the EU any more. Turkey is abandoning some of the building blocks it fashioned in the hope of a more prosperous, European future: freedom of expression is under attack again, the judiciary is under assault by the government, respect for contracts and the rule of law is slipping, educational achievement remains shockingly low, the 31-year-old insurgency in the Kurdish east is heating up again, and President Erdoğan is wrapping himself in an ever-more authoritarian cloak.

Nevertheless, anyone can still drink rakı on the banks of the Golden Horn. Surrounded by hundreds of others who are enjoying the balmy evening we toast Turkey as the waiter puts a big tray of exquisitely grilled fish on the table. He turns out to be a migrant newly arrived from the southeastern Syrian border city of Mardin and is very happy to have found this job. It won’t be for long, however, he predicts with a certain gloom.

A perpetual roller-coaster of change

Two mornings later I walk down the hill one last time. From a distance I already see the little wooden fishing boats and metal passenger ferries bobbing up and down by the quayside. To my shock, however, the sea front is covered in splintered tables and chairs, bulldozer-churned earth and piles of broken plates. All the restaurants, impromptu kitchens and subdivisions have been smashed. A nearby line of fish merchants, who for as long as I can remember have been selling fresh fish to Istanbullus catching the ferry to the Asian side, have also been levelled. Scruffy kids already are scavenging for anything of value that is left.

“What happened?” I ask one of the plain-clothes young men, apparently from the municipality, who are putting up a fence around the devastated scene.

“We’re cleaning it up, making it better, proper,” he replies.

Another tells the man not to bother with me. “He’s some dirty foreign agent.”

My cheeks and ears burn at the old insult and I stalk off towards the Galata bridge and its warren of shops. I’m also filled with indignation over the random way that the lightning bolt of state-sanctioned destruction has hit a part of town that has given me so much pleasure to live in.

“What on earth happened to the fish market and the restaurants?” I ask the owner of a shop selling batteries, radios and highly realistic air pistols.

“Thank God they took care of those fly-by-night places at last!” he replies. “Those eastern mafia guys were taking over. It was putting tax-payers like us out of business. The state should show who’s in charge.”

Even if I’m pretty sure that in this case the “state” is just the municipality seizing control of a lucrative piece of territory that will be handed out to its own partisan supporters, any anger I have soon seems pointless. A few more exchanges with phlegmatic shopkeepers along the way persuade me to view the debris as typical of a city that is constantly breaking and reinventing itself.

This perpetual roller-coaster of change is part of what makes life in Turkey both exhausting and addictive. By the time I’m back up the hill and home, I am reconciled that there will always be another new pop-up restaurant to try out on some unmarked Istanbul quayside. The trick is only to be able to find it in time.

Note: An earlier version of this article said that Galatasaray won the European Championship in 2000. In fact it won the UEFA Cup. This is the secondary cup competition in Europe behind the Champions League, and used to be called the European Champion Clubs’ Cup or European Cup. Many thanks to Alexis Rowell!