It doesn’t take long after I drive out of the sleek OR Tambo international airport for the penny to drop. Again. Johannesburg is the bastard child of the worst aspects of capitalist greed and 20th-century racism. Nearly 150 years after its formation, this sprawling metropolis is still scarred by the sins of its genesis.

Even with the explosive rise of the black middle class, the presence of blacks in formerly white suburbs remains low

Johannesburg – like Cape Town, Durban, Port Elizabeth and other cities in South Africa – is visibly and traumatically segregated. These remain cities divided.

The rich of Johannesburg still live in the sumptuous northern suburbs, where the food at some restaurants is Michelin-star quality and house prices are eye-watering. These areas remain largely white, although that is changing at a glacial pace. The workers are in Soweto and Alexandra and other poor, crime-plagued black enclaves. It has always been this way with Johannesburg, and it remains pretty much as divided 25 years after apartheid collapsed and 29 years after Nelson Mandela walked out of prison.

This economic powerhouse is Africa’s city of dreams – and nightmares. Its population of nearly 10 million are drawn from all corners of South Africa and increasingly from Zimbabwe, Nigeria, Malawi and Bangladesh. The city remains a magnet for those hoping for a better life.

Q&A What is South African cities week? Show Hide Twenty-five years after the fall of the brutal apartheid regime, South Africa's cities remain hugely divided, both economically and racially. This week Guardian Cities explores the incredible changes taking place, the challenges faced and the projects that bring hope. Africa correspondent Jason Burke reports from the Flats, where violence and death are endemic just miles from Cape Town's spectacular beaches and trendy cafes. Author Niq Mhlongo pens a love letter to the "other Soweto", one that visitors to gentrified Vilakazi Street never see. We hear from Port Elizabeth, where one architect is using recycled materials to transform his city, and Durban, where a surf school is changing the lives of vulnerable children. We explore the deadly underground world of zama zama gold miners operating illegally under the city of Johannesburg, visit the Afrikaner-only town of Orania and publish an extraordinary photo essay by Magnum nominee Lindokuhle Sobekwa, who documents life in a formerly white-dominated area where his mother once worked as a domestic helper. Nick Van Mead

It is unique as the only major city in the world not built by the seaside or on the banks of a major river. This is because it is the child of gold, not trade. Just a patchwork of farms when gold was discovered in 1884, it swiftly transformed into a chaotic, violent concatenation of settlements that attracted white adventurers, gold diggers (literal and figurative), sex workers, settlers, criminals, shysters, black labourers and elites from around the globe – all looking to make a fortune.

Facebook Twitter Pinterest People queue to cast their votes at a polling station in Soweto in April 1994, in South Africa’s first all-race elections. Photograph: Denis Farrell/AP

It mutated into a frontier town and grew in a colonial fashion – blacks and whites remained largely separate, with the white mine-owners building mansions that sprawled into rich northern suburbs while black people were pushed to the south into townships.

Apartheid formalised the loose colonial arrangement in the 1940s, creating a black labour reserve named Soweto (from South Western Townships) and banishing black people from the city while forcing them to carry a dompas (permit) at all times to show cause to be there. For 46 years from the formal introduction of apartheid in 1948 until its demise in 1994, this was the architecture of apartheid Johannesburg. Separate and unequal; black and white; rich and poor.

Then 1994 happened. Mandela and his party, the ANC, were installed in office. Hopes for a new South Africa and a new Johannesburg – integrated, non-racial and free of the divisions of the past – were high. Spatial apartheid would be done away with thanks to creative and determined urban planning.

Facebook Twitter Pinterest The Maboneng development zone in Johannesburg in 2013. Photograph: Bloomberg/Getty Images

It has not quite happened. In my neighbourhood of Parkview, a tree-lined middle-class suburb in the jacaranda tree shadow of the “Randlords’” mansions in Westcliff, mine remains one of woefully few black families. Even with the explosive rise of the black middle class in the mid-2000s, the presence of black people in formerly white suburbs across Johannesburg remains low.

There is a reason for this glacial pace of change. Johannesburg is a microcosm of South Africa. The World Bank said in May 2018 that South Africa remains the most economically unequal country in the world. Poverty levels are highest among black people. Whites make up the majority of the elite or top 5% of the population. Hence the stubbornness of spatial segregation.

After the collapse of apartheid, Mandela and his new team vowed to provide housing, water, electricity and other amenities to the previously disadvantaged. They didn’t expect to get such a huge influx of new residents into the cities. Since 1994, millions of people have set up shacks on the peripheries of townships and cities across the country.

The response has been to rush to these unplanned new peripheral areas – some built on dangerous river banks – and build formal, though tiny, houses. The result is massive new low-cost housing units on the peripheries of cities and very little or no deliberate urban planning that leads to integrated housing solutions. The rich stay in the rich suburbs while the poor join other poor people on the periphery.

Facebook Twitter Pinterest A pedestrian passes by a residential construction project in Maboneng. Photograph: Bloomberg/Getty Images

Hopes for the future

It doesn’t mean that there is no change. In 2016 the government statistician published a series of maps that illustrate Johannesburg is the most integrated city out of its six major metros. Encouraging as that picture is, it is also problematic. The Johannesburg central business district has a high percentage of black African residents – but the past 20 years have been characterised by “white flight” into the northern suburbs. Johannesburg’s townships, like Soweto, remain largely disconnected from business districts and formerly white suburbs, despite initiatives such as bus rapid transit to make it easier for Soweto residents to get to work in formerly white areas.

Even more hopeful is the decision this February by the city of Johannesburg to adopt a first-of-its-kind inclusionary housing policy that compels private developers to make 30% of the homes in all future residential developments affordable, regardless of where they are built. Implemented properly, it could be a game-changer for the city.

Many of my favourite parts of Johannesburg have been facilitated by the Johannesburg Development Agency, together with a handful of savvy and brave private developers. The Newtown Cultural Precinct in the CBD is a fine example, incorporating commercial developments and sleek affordable housing.

Johannesburg's new policy to make 30% of homes in future developments affordable could be a game-changer for the city

Possibly the trendiest part of the old Joburg CBD is Maboneng, a collection of 55 buildings bought and restored by the JDA in partnership with entrepreneur Jonathan Liebmann. The company Liebmann founded collapsed earlier this year, with units being auctioned off at way below estimated market value to bargain-hunters. But it is still a hive of restaurants, hotels, residential flats and a major artistic hub, with international artist William Kentridge as a tenant. However there are others springing up all over town.

Yet Johannesburg’s fortunes are intricately intertwined with South Africa’s – and the country has gone through a turbulent 10 years under the leadership of ousted former president Jacob Zuma. Now led by former trade unionist and businessman Cyril Ramaphosa, the country is battling to deal with the corruption that thrived under Zuma. Ramaphosa says all the right things, as he did in London this week, but internal ANC politics stop him from introducing vigorous economic reforms to kickstart the economy in a country where unemployment is now just under 30% and government finances are deteriorating rapidly.

Waving not drowning: the street children finding refuge in Durban's surf scene Read more

Young people are restless and are becoming increasingly disillusioned with politics. The number of South Africans under 20 who registered to participate in May’s general election was the lowest since at least 1999, data from the Independent Electoral Commission showed. Among citizens aged 18 to 29 – the biggest segment of the voting population – registrations are at their lowest in at least a decade.

The frustration of youth is palpable. Every morning traffic reports warn of protests by young people blocking major roads with burning tyres and rocks to demand services and jobs. It is a ticking timebomb.

Yet there is a sense in South Africa that things can be turned around. That would give our cities a shot at becoming more inclusive, more liveable and more human.

• Justice Malala is an award-winning journalist, television host, political commentator and newspaper columnist. His book on South Africa’s transition from apartheid to democracy will be published in the US next year.

Follow Guardian Cities on Twitter, Facebook and Instagram to join the discussion, catch up on our best stories or sign up for our weekly newsletter. Share your views here on how South African cities have changed in the last 25 years

