"Cities are our greatest creation," states PD Smith in his introduction to this richly packed, colourful and well-written primer on the role the city plays in our lives. From the outset he reminds us: "Today, for the first time in the history of the planet, more than half the population – 3.3 billion people – are city dwellers … by 2050, 75% will be urbanites."

In keeping with the spirit of the city itself – more a vital and unpredictable organism than a rationally planned machine for living in – this book has been planned as a walk, or wander, through the city with no set routes or strict chronology. "As in a real city," says Smith, "you can follow any number of pathways through this book. And don't worry about getting lost. Some say it's the only way really to experience a city." Smith is right when he talks about getting lost, for there is always another city alley to take, doorway to enter, park to stroll through or some overlooked or even forbidden quarter of a city to sidle through. The greatest cities are inexhaustible, and not least because they are constantly changing. And when a city stops evolving, its lifeblood freezes and it becomes – as history proves – little or nothing more than a museum showcasing its own past or a cluster of haunting ruins.

And, yet, ambitious city builders, from ancient Sumeria to Shanghai today, have nearly always thought of the city as a rational and thoroughly planned ideal brought to life in avenues and public squares arranged as straight as a die. One of Smith's brief and illuminating chapters looks at the idea of the "Ideal City", spinning through notions conjured by Plato, Vitruvius, Leonardo, Thomas More, Campanella, Ebenezer Howard and Le Corbusier. As a counterpoint, he also cites Italo Calvino's Marco Polo, who in that enchanting book Invisible Cities tells Kublai Khan: "Cities, like dreams, are made of desire and fears, even if the thread of their discourse is secret, their rules are absurd, their perspectives deceitful, and everything conceals something else."

The city, then, is never as rational as its founders, patrons, architects, planners, bureaucrats and engineers might have wished it to be. Truly great cities have always been a heady mix of the planned and the unplanned, the rational and the irrational, the dreamlike and the matter-of-fact. A great city today might have a magnificent core of grand central streets, stirring architecture, a comprehensive public transport system running like clockwork, secret sewers going about their sulphurous business untiringly, sane governance, bright schools, comforting hospitals, and all of these underpinned by healthy commerce and adorned by a confident culture. And, yet, the same city would be woven through with the unpredictable worlds of fashion, music, art, cuisine, carnivals, hobbies, cults, clubs.

So behind the walls of the city – Smith has a chapter on these – there is darkness, graffiti, street language, uprisings, religions, ghettos and slums, cathedral-like railway stations, traffic, trade, bazaars, malls, museums, red-light districts and so much else. Smith packs the blood, guts, underbelly and driving forces of the archetypal city into chapters as densely packed as the streetscapes of Manhattan or Hong Kong. He reminds us that, today, such dense cities are surrounded by ever-expanding and mind-numbingly banal outer-suburbia; as a consequence – in the words of the theorist of renewable energies, Peter Droege – the promiscuously sprawling 21st-century city is "a fossil-fuel construct in search of rapid restructuring".

Smith's lively urban tutorial is framed between chapters looking at the origins and the future of the city. I was touched to see him so confident that the world's first city might well have been Eridu, in what is today southern Iraq, as this brought back memories of my getting to this antique city buried in scorching desert sands a few months before George W Bush and Tony Blair launched their assault on the venerable, if much younger, city of Baghdad. I am still researching the world's earliest cities and half expect to find examples older than Eridu. But what Eridu taught me in a direct way, as its hot sands slipped through my hands, was that cities are indeed organisms: they are born, mature, grow old and sometimes become irrelevant.

The stuff of lofty intentions and grubby backstreet life, the city represents much of our restless and contradictory natures. "In this dynamic, cosmopolitan space," Smith writes, "lies the wellspring of our creativity as a species. The greatest cities nurture and stimulate ideas in science and the arts that are the very heart of human civilisation. For this reason, sustainable, humane and well-governed cities are our best hope for the future."

Amen. Although, Smith might have added that to keep them alive, and healthy and "civilised", we will have to wrestle with the forces of globalised banality threatening to turn cities into homogenous and mindless machines for unsustainable consumption. The sorcery of cities should not be lost; Smith's ebullient guidebook helps to remind us why.

• Jonathan Glancey's Nagaland is published by Faber.