She stumbles through the city streets, her eyes now unfocused – absorbing a confusion of grey, brown and red parallelograms, that she knows to be crushingly weighty, and yet which she feels to be as insubstantial as dandelion spores – and now locked on the faces of those others who pass by her with such fixity that, if she concentrates on a single physiognomy, she senses that with only a little further effort she would be able to deduce everything about that individual: his age, occupation, sexual history, political affiliations, the names of his family and friends. For milliseconds she is transfixed by the uniqueness of his personality – and then he is subsumed once more into the crowd. In the roadway beside her the traffic courses: buses whoosh, lorries grind, cars bounce, motorcycles swerve – yet there is no mechanical noise at all, these steely parallelograms interleave, shuffle and montage to the accompaniment of electronic peeps, beeps, burbles and thrums – a soundtrack that our walker can choreograph all the traffic to, human and vehicular, her deft, darting eyes seamlessly stitching order out of the chaos so that everything around her skips to her divinely ordained beat. She is completely lost: she could not tell you the name of a single street or notable building.

She is disoriented – and yet her progress is a perfectly plotted trajectory through urban space: she looks into the glowing multifaceted jewel in the palm of her hand and here other parallelograms interleave, shuffle and montage in response to the tweezer motions and baton-flicks of her fingers. It tells her where to go, the jewel, and when she places it to her ear it speaks to her, so that in turn she can command her own faltering legs to carry her to the right, to the left, straight ahead, until at last there is a face she recognises – or does she? Unbidden, his age, his occupation, his sexual history, his political affiliations, the names of his family and friends all come to her. And yet in the milliseconds before they intercept one another she is transfixed by an awful sense of the stereotypy of his personality – then he is mercifully released from the crowd and into her arms. "Sorry I'm late," she puffs – she knows she is precisely three kilos overweight. "It was miles to get here from the tube." She has, in fact, walked exactly 723 metres.

❦

I hope the above will be taken for what it is: a lightly poeticised account of the mental state of an average young woman negotiating her way through the urban environment. Her responses to her fellow city dwellers, to the road traffic, to the business of finding her way using a handheld GPS system while listening to music on her MP3 player are all quite normal, and yet, set down like this they seem to me to be indisputably analogous to a clinically defined psychotic state. Like a sufferer from psychosis, our young woman's conception of reality radically diverges from her environment: she is surrounded by actual buildings, with a defined and apprehensible nomenclature; the people she passes are neither clones nor individually known to her but a mass of strangers; neither these people, nor their vehicles are moving in synch with the music she listens to; and finally: her perception of distance is distorted, while her ability to negotiate her environment is dependent on systems external to her own mind that, for all their technical efficacy, are as opaque to her as the magical rituals of a shaman. Indeed, so long as the rendezvous with her boyfriend is made, it would make no difference to our young woman if it were effected by her consulting a fetish, or flinging a handful of bones on to the pavement and directing her footsteps in accordance with their arrangement.

That our mode of being in the industrialised – and now post-industrial – built environment is in some sense profoundly awry is by no means a new observation. Writing in the 1840s, Friedrich Engels notes the "brutal indifference, the unfeeling isolation of each in his private interest", which "becomes the more repellent and offensive, the more these individuals are crowded together in a limited space". What he characterised as the fundamental principle of society everywhere, was nonetheless "nowhere so shamelessly barefaced, so self-conscious, as just here in the crowding of the great city".

But writing 20 years earlier, Thomas de Quincey had already perceived in the great moiling of London's thoroughfares a fundamental alteration in the nature of human connectivity. As a teenage runaway he was, by his own account, saved from starvation on the streets by a young girl, Ann, with whom he spent some weeks. On their parting he arranged to try to meet with her on the corner of Titchfield Street at a given hour in the evening. If either of them failed to make this rendezvous they were to try again the following evening – but she didn't come after many nights, and although he searched for her throughout the city, De Quincey was never to find her again. He says: "This, amongst such troubles as most men meet in life, has been my heaviest affliction. If she lived, doubtless we must have been sometimes in search of each other at the very same moment through the mighty labyrinths of London."

You might argue that had he and Ann had access to the web, they could have zeroed in on each other unerringly; that the mighty labyrinth of London – which at that time had a population of around an eighth of what it is today – would have been rendered as flimsily irrelevant as the walls that are crashed through by the protagonists of mobile phone TV adverts, designed expressly to demonstrate that the urban environment's complexity is annulled by the juju of technology. De Quincey says his failure to find Ann was his "heaviest affliction", and in this he seems to make his loss emblematic of an entire society's abandonment of personalised bonds. He asks numerous people for Ann's whereabouts, but in the city all that counts – that is measurable – is the crowd; the individual – especially the poor, female individual – counts for nought.

In Edgar Allan Poe's short story "The Man of the Crowd", published in 1840, the unnamed narrator is initially transfixed by a particular physiognomy: "a countenance which at once arrested and absorbed my whole attention, on account of the absolute idiosyncrasy of its expression. Any thing even remotely resembling that expression I had never seen before. I well remember that my first thought, upon beholding it, was that Retzch [sic], had he viewed it, would have greatly preferred it to his own pictorial incarnations of the fiend." You don't have to be familiar with the engravings of Moritz Retszch (1779-1857) to get what Poe is driving at, especially given that up until this point the faces of those the narrator has passed by in the teeming London streets have been described solely as different versions of types – the eponymous Man of the Crowd is the first individual he has chanced on.

However – and here Poe's irony dramatically prefigures all of the 20th century's sense of urban alienation – as the narrator follows this singular person, he becomes slowly but insistently aware that his quarry is unable to exist apart from the crowd, that the haunted, fiendish expression and threadbare appearance is a direct product of a need this man has always to be among others. Poe's epigraph for "The Man of the Crowd" is from La Bruyère: "Ce grand malheur, de ne pouvoir être seul" (it is the greatest misfortune to be unable to be alone). Some might argue that the city presents us with a liberating experience, that by freeing us from the oversight – moral and political – of small, enclosed communities, the urban existence fostered self-actualisation and, by extension, good mental health. But the examples of Engels, De Quincey and Poe belong to an era of rapid urbanisation when the cityscape was still traversable on foot. Is it too fanciful to suggest that the genius of these early 19th-century writers was to anticipate the western conurbations of the early 21st century, where oversight of the crowd is undertaken by closed circuit TV systems only sporadically and randomly monitored, while distance and orientation are abstracted from the physical plane?

I think not. Poe's unnamed narrator is unable to be alone in a social sense – unable to experience solitude, depending for his very character on modern mass urbanity. Our walker, by contrast, is unable to experience being alone in place itself: not knowing where she is, and too unfit to travel across appreciable portions of the city by her own motive power, she is condemned to a socialised spatial existence.

Rebecca Solnit in Wanderlust, her magisterial history of walking, writes of her own experiences of danger during nighttime promenades in San Francisco: "I was advised to stay indoors at night, to wear baggy clothes, to cover or cut my hair, to try to look like a man, to move someplace more expensive, to take taxis, to buy a car, to move in groups, to get a man to escort me – all modern versions of Greek walls and Assyrian veils." And so she realises that "many women had been so successfully socialised to know their place that they had chosen more conservative, gregarious lives without realising why. The very desire to walk alone had been extinguished in them …" She later observes that "Black men nowadays are seen as working-class women were a century ago: as a criminal category when in public."

The Stoic philosopher Epicurus maintained that free will was only an illusory sense we experience when the actions necessitated for us by circumstance fortuitously coincide with what we happen to want – it's my belief that this perfectly characterises the psychotic spatial awareness of the vast majority of contemporary urban dwellers; while the existential threats afflicting women, and the state-sanctioned ones that impinge, in particular, on young black men in British cities, have been internalised even by those – the white, the middle-aged and the middle class – who have no reason to be so trammelled. Put bluntly: deprived of mechanised means of locomotion – the car, the bus, the train – and without the aid of technology, the majority of urbanites, who constitute the vast majority of Britons, neither know where they are, nor are capable of getting somewhere else under their own power.

Nor even yet are they able to formulate the desire to do such a thing. So far as they are concerned, the journeys to work, to shop, to be entertained, to liaise with their social circle are all the utilisation of the built environment – such unpremeditated and willed walking as there is remains within these contexts, the most egregious example being the shopping mall itself. Yet a little over a century ago, 90% of Londoners' journeys under six miles were still made on foot – many of these would have been commutes, but even a walk to work involves a physical possession of the built environment and the exercise of orienting skills.

Year on year, the number of journeys taken on foot declines – indeed, on current projections walking will have died out altogether as a means of transport by the middle of this century. No longer subjected to the measure of man – or woman – or her oversight, the city has already acquired distorted lineaments: vastly extended thoroughfares are lined by cul-de-sacs, while the architecture defined by Rem Koolhaas as "junkspace" presumes that only a corridor can be a viable destination – especially if it has a cash machine. Suburbia, and the inter-zone between the city proper and its rural hinterland, is the tangible form of this disregard, being a collection of locations that no longer convey any sense of place.

Recall Borges's famous "Map of the Empire whose size was that of the Empire … In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography."

Borges's animals and beggars are those who still seek the disciplines of physical geography – we understand that to walk the city and its environs is, in a very powerful sense, to use it. The contemporary flâneur is by nature and inclination a democratising force who seeks equality of access, freedom of movement and the dissolution of corporate and state control.

This is an edited version of Will Self's inaugural lecture as professor of contemporary thought at Brunel University.