This Thursday Londoners will vote to elect a new city mayor, and I thought it would be a good moment to put up a great documentary about how Norman Mailer stood for Mayor of New York in 1969.

It is a lovely film - directed by a brilliant documentary maker called Dick Fontaine, and beautifully shot in the fluid way of cine-verite back then. But more than that it captures the rise of a phenomenon that has come to dominate (and possibly strangle) western metropolitan society today. It is the rise of the hipster. By this I don't mean the present cliche of the ironic moustaches that live in Hoxton and Hoboken - but a new cultural elite that was beginning to emerge at that time, the rebellious, stroppy bohemians who looked to culture rather than politics to define their identity - and above all their difference from others.

In the film you can see them peeking through in the backdrop of many of the scenes, for Mailer attracts them. In his perverse individualism and rebelliousness - one of his slogans is "We're no good, and we can prove it" - Mailer captured a new sensibility. This was because he combined a revulsion against a tired old culture together with a distrust of the political system, and the hipsters loved it.

But Mailer was a complicated man - and as well as embodying many of the hipster values he was also a perceptive and vocal critic of the new sensibility. Back in 1957 he had written an essay for Dissent magazine called The White Negro. In it he had described how fears of nuclear annihilation had begun to produce a new kind of young alienated being in America. These hyper-individualists trusted only their own feelings and desires and refused to be part of any group or organisation. And in black culture, Mailer said, they found their identity - the culture of the dangerous outsider.

This outsider culture had originally been created, Mailer wrote, by blacks in response to racial oppression and violence. But for the "white negroes" that culture was then co-opted in order to give a meaning and grandeur to their psychopathic narcissism:

"In such places as Greenwich Village a menage-a-trois was completed - the bohemian and the 'juvenile delinquent' came face to face with the Negro, and the hipster was a fact in American life. If marijuana was the wedding ring, the child was the language of Hip for its argot gave expression to abstract states of feeling which all could share, at least all who were Hip. And in the wedding of the white and the black it was the Negro who brought the cultural dowry.

So there was a new breed of adventurers, urban adventurers who drifted out at night looking for action with a black man's code to fit their facts. The hipster had absorbed the existentialist synapses of the Negro, and for practical purposes could be considered a white Negro."

Mailer also pointed out that this new breed of "psychic outlaw" could be equally a candidate for the most reactionary or the most radical of political movements. And in the film there is a fascinating scene where Mailer takes on the trades unions on one of the avenues in New York. He tells them that in the past they were a heroic movement - but that now they have become a repressive, stultifying force in society - in particular in the way they are refusing to allow blacks and hispanics to move up society. It is an odd moment because as you watch you realise that it was elements of this rebellious individualism that both Thatcher and Reagan would later harness. And that possibly, if the left had got hold of it earlier, then the history of the West might have been very different.

In the 1970s the phenomenon that Mailer had identified grew massively. And as it did the new cultural bohemians co-opted another outsider culture to give themselves further identity - the gay culture that had risen up in response to homosexual discrimination. Then in the 1980s that bohemian individualism became the driving force that permitted consumer capitalism to reinvent itself - because it offered the ever-multiplying hipsters the objects through which to express their rebellious difference.

Today it is possible to argue that we have all become gay white negroes. We all listen to "edgy urban" music, spend our time in the gym, go shopping and groom ourselves, take lots of drugs, have sex and then spend the rest of the time talking to our friends about the impossibility of finding real love and connection in the world.

Far from an expression of rebellion it has become the conformity of our age. And if Mailer was right - that it was a sensibility originally born out of the existential fears of nuclear holocaust - then it lost its real radical purpose when the cold war ended.

Instead the white negro hipster has actually become one of the central conservative pillars of our time - because their real function is now simply to prop up an increasingly shaky system of credit and rolling consumption.

And one wonders where are the real outsiders of out time? Who are the new "white negroes" of our age?