Continuing our critiques of cities by writers who live in them, would Australia's largest metropolis actually benefit from ditching its famous landmarks and concentrating on the rest of it?

Paris, Rio, Venice, Cape Town; some cities conjure up an instant image, either natural or manmade, for people who have never been there. Others, even very large ones, have no such handy reference point. How much more difficult is it for most outsiders to picture Turin, Guangzhou or Santiago?

Sydney is one of the most recognisable places on Earth, thanks to its famous city beaches and the two contrasting works of genius on the harbour, the magnificently functional bridge and the maddeningly dysfunctional opera house.

I can't recall hearing Sydneysiders complain about its natural beauty or its manmade wonders – like people in most tourist-friendly cities they are more inclined to regret that they don't take advantage of the obvious attractions on their doorstep more often. But I often wonder whether such brazen advantages also breed a certain unconscious complacency that makes Sydney a worse place to live than it might be if it had to work with the raw material of, say, Warsaw or Madrid. Or Melbourne.

Let's keep this in perspective. For most of its inhabitants, Sydney is a wonderful home, regularly featuring among the top 10 in the (admittedly dubious) lists of the world's most liveable cities. But its glaring inadequacies are rendered somehow more infuriating and intractable by the hackneyed image it offers to visitors. Tourism ads for Sydney almost invariably feature the landmark sites and little else. Contrast this with Melbourne, where visitors are obliquely enticed by the prospect of meeting sexy hipsters by going on treasure hunts and trailing giant balls of string through the city's intriguing lanes. Its appeal is the quirky and unexpected, not the ragingly obvious (contrast also with South Australia , which seems keen to attract a niche market of visitors who enjoy gothic fantasies about sex, food and death – but I digress).

Bondi Beach – Sydney tries to appeal to tourists who want to do 'obvious' things like see the harbour and lie on the beach, rather than those seeking quirky or unusual sights. Photograph: David Messent/Getty Images

The harbour is a blessing for Sydney's modelling shots of itself and for New Year's Eve fireworks but a curse for urban planners. Sydney's history and topography ensured that its commercial and political centre would be dramatically skewed towards the east, a good 12 miles (20km) from the geographical centre of the city's population (currently lying somewhere around the suburb of Ermington, near Parramatta). Both intangible sources of power and tangible public services, such as railway lines, are funnelled inexorably into the narrow peninsula that runs down to Circular Quay.

As the harbour lost its industrial significance, formerly working-class suburbs near the water rapidly became highly desirable objects of gentrification, an extreme example of the process familiar to so many other cities in the developed world. Several factors lie behind Sydney's insane property prices , but they reach their most wallet-dominating levels wherever water views, or even glimpses, can be offered. While recent development policy has encouraged the proliferation of apartments along transport routes and in suburbs closer to the city, Sydney is still about half as densely populated as London, and continues to sprawl to the north-west and south-west. If anything, the longstanding divide between east and west is only sharpening as it does so. It's not a simple matter of rich and poor.

Andrew Merry's extraordinary aerial photographs of new suburban developments on the western fringes reveal the huge size of some new houses (as well as the terrifying environmental potential of treeless streets, relatively tiny back gardens and car-only access). But there is no doubt the west as a whole feels ignored and patronised by the more scenic parts of the city. This feeling has recently been manifested in football, of all things, through the creation in 2012 of a new team for Western Sydney in Australia's A-League.

It has fed ravenously on hostility to the earlier established Sydney FC (based well and truly in the east) and its reputation for flashy arrogance. At one of the first derbies between the two, the Western Sydney fans held up signs marked simply with their postcodes, the Sydney Morning Herald noted, as "a cheeky pointer to places many Sydney FC fans have never visited". The city's political arrangements do little to overcome that divide and foster unity.

Sydney harbour bridge under construction, in 1930. Photograph: Popperfoto/Getty Images

The responsibilities of the New South Wales state government naturally extend beyond Sydney, while the grandly titled lord mayor of Sydney presides over what is essentially a tiny local council, but one that includes the central business district. The current mayor, Clover Moore, a canny independent who has cultivated support among the gay community and other groups easily stereotyped as "inner city elites", is an object of toxic hatred for rightwing radio hosts and Sydney's News Corp-owned tabloid, the Daily Telegraph. Their fury has been directed particularly at Moore's efforts to construct bike paths throughout the city, a classic issue that is assumed to divide the largely car-dependent west from inner-city trendies.

The lack of an overarching authority for the city may be one reason for its atrocious decisions on transport, which has left large parts of the suburbs (predominantly in the west) with very poor access to public options. A smartcard similar to London's Oyster, unifying ticketing systems across trains, buses and ferries, was first announced in 1996, to be in place for the 2000 Olympics. The version that actually works is finally being rolled out this year.

The new federal government under Tony Abbott has slashed funding for new public transport projects, while encouraging more road building, meaning Sydney is likely to go further down the path of toll roads with charges that pay for their construction, rather than to manage the traffic flow – inner Sydney has no congestion charge, but you have to pay to use tunnels that keep cars out of the city streets.

The acclaimed Danish urban designer Jan Gehl has said Sydney is full of "good examples of what not to do" to make a city centre that is less abrasive and unforgiving to the people who live and work there. But even he has offered fewer ideas on how to turn attention away from the transfixing pull of the harbour.

Would it be easier to make Sydney a more coherent and less frustrating place if it was in a less glamorous location? It seems illogical but it's hard to avoid the sense that cities without such obvious attractions make greater efforts to get the best out of what they do have.

They may not be such exciting places to visit but good planning, innovation, attractive public spaces, equality of treatment and decent transport matter more to the people who live there. Not convinced? Try asking someone from Melbourne.

• This article was amended on 18 February 2014. An earlier version had a picture caption which said the picture showed the harbour bridge under construction in 1920.