Francis Fukuyama's influential essay 'The End of History?' announced the triumph of liberal democracy and the arrival of a post-ideological world. But was it just a rightwing argument in disguise? And has the demise of utopianism ushered in a 'sad time'?

In the summer of 1989, the American magazine the National Interest published an essay with the strikingly bold title "The End of History?". Its author, the political scientist Francis Fukuyama, announced that the great ideological battles between east and west were over, and that western liberal democracy had triumphed. With anti-communist protests sweeping across the former Soviet Union, the essay seemed right on the money. Fukuyama became an unlikely star of political science, dubbed the "court philosopher of global capitalism" by John Gray. When his book The End of History and the Last Man appeared three years later, the qualifying question mark was gone.

The "end of history" thesis has been repeated enough to acquire the ring of truth – though it has also, of course, been challenged. Some critics have cited 9/11 as a major counterexample. Others have pointed to the rise of Islamic fundamentalism and the Arab spring as proof that ideological contests remain.

But Fukuyama was careful to stress that he was not saying that nothing significant would happen any more, or that there would be no countries left in the world that did not conform to the liberal democratic model. "At the end of history," he wrote, "it is not necessary that all societies become successful liberal societies, merely that they end their ideological pretensions of representing different and higher forms of human society."

Fukuyama was talking about ideas rather than events. He believed that western liberal democracy, with its elegant balance of liberty and equality, could not be bettered; that its attainment would lead to a general calming in world affairs; and that in the long run it would be the only credible game in town. "What we are witnessing," he wrote, "is not just the end of the cold war, or a passing of a particular period of postwar history, but the end of history as such: that is, the end point of mankind's ideological evolution and the universalisation of western liberal democracy as the final form of human government."

Fukuyama drew on the philosophy of Hegel, who defined history as a linear procession of epochs. Technological progress and the cumulative resolution of conflict allowed humans to advance from tribal to feudal to industrial society. For Marx, the journey ended with communism; Fukuyama was announcing a new destination.

For a long time his argument proved oddly resilient to challenges from the left. Neoliberalism has been pretty hegemonic. Over the last three years, however, in a belated reaction to the 2008 bank bailouts, cracks have started to appear. Global Occupy protests and demonstrations against austerity have led many commentators on the left – including the French philosopher Alain Badiou in The Rebirth of History and Seumas Milne in his collection of essays The Revenge of History – to wonder whether history is on the march once again. "What is going on?" asks Badiou. "The continuation, at all costs, of a weary world? A salutary crisis of that world, racked by its victorious expansion? The end of that world? The advent of a different world?" He tentatively regards the uprisings of 2011 as game-changing, with the potential to usher in a new political order. For Milne, likewise, developments such as the failure of the US to "democratise" Iraq and Afghanistan, the financial crash and the flowering of socialism in Latin America demonstrate the "passing of the unipolar moment".

What remains an open question is whether these developments – dramatic as they are – will actually result in anything. Leaderless and programme-light, dissent keeps failing to cohere, fragmenting into online petitions and single-issue campaigns. Is the left going to mount a coherent ideological challenge to the right, or are these just border skirmishes? Has history ended, or not?

As some on the left have long realised, Fukuyama was performing an ideological sleight of hand. Is "western liberal democracy", as he argued, really an application of the principles of the French revolution? Or is it in fact a way of cloaking rightwing politics in benignly incontestable disguise? "Man's universal right to freedom" sounds inspiring, but if you are on the right it is another way of saying economic liberalism. Besides, even that is a fiction: capitalism pretends to love free markets; in reality, it rigs markets for elites.

When he wrote "The End of History?", Fukuyama was a neocon. He was taught by Leo Strauss's protege Allan Bloom, author of The Closing of the American Mind; he was a researcher for the Rand Corporation, the thinktank for the American military-industrial complex; and he followed his mentor Paul Wolfowitz into the Reagan administration. He showed his true political colours when he wrote that "the class issue has actually been successfully resolved in the west … the egalitarianism of modern America represents the essential achievement of the classless society envisioned by Marx." This was a highly tendentious claim even in 1989.

In 2006, in the wake of George W Bush's catastrophic blunders in Iraq and Afghanistan, Fukuyama repudiated neoconservatism in a book titled America at the Crossroads. In order to keep his end-of-history thesis intact, Fukuyama argued that the neocons had gone off on a Leninist tangent of historical determinism and artificial nation-building, and had departed from the correct understanding of historical evolution as an organic byproduct of material comfort and access to consumer goods.

The "post-ideology" sleight of hand nevertheless continues. "The markets", which he hailed as the engine of progress, were and are talked about as "natural" – as if they were forces of gravity or Darwinian evolution. They are believed to impose "realistic" limits on policy; political prioritising hides behind practical references to the "public purse". "This is the sober reality I must set out for the country today," David Cameron said in June 2010, announcing his plan for cuts in public spending. "We are not doing this because we want to, driven by theory or ideology … We are doing this because we have to." Through three decades of wonkery and spin, the right has systematically constructed an ideological movement that presents itself as anything but systematic, anything but ideological.

Fukuyama distinguished his own position from that of the sociologist Daniel Bell, who published a collection of essays in 1960 titled The End of Ideology. Bell had found himself, at the end of the 1950s, at a "disconcerting caesura". Political society had rejected "the old apocalyptic and chiliastic visions", he wrote, and "in the west, among the intellectuals, the old passions are spent." Bell also had ties to neocons but denied an affiliation to any ideology. Fukuyama claimed not that ideology per se was finished, but that the best possible ideology had evolved. Yet the "end of history" and the "end of ideology" arguments have the same effect: they conceal and naturalise the dominance of the right, and erase the rationale for debate.

While I recognise the ideological subterfuge (the markets as "natural"), there is a broader aspect to Fukuyama's essay that I admire, and cannot analyse away. It ends with a surprisingly poignant passage: "The end of history will be a very sad time. The struggle for recognition, the willingness to risk one's life for a purely abstract goal, the worldwide ideological struggle that called forth daring, courage, imagination, and idealism, will be replaced by economic calculation, the endless solving of technical problems, environmental concerns, and the satisfaction of sophisticated consumer demands."

It is hard not to conclude that this passage offers an accurate portrait of our age, in which the campfire conversations of young activists merely concern relative concentrations of CO2; the politics of nudge and solutionism are embraced by right and left alike; and the hordes camped out on the streets of Rio de Janeiro are awaiting the opening of Latin America's first Apple store.

"In the post-historical period," Fukuyama continues, "there will be neither art nor philosophy, just the perpetual caretaking of the museum of human history. I can feel in myself, and see in others around me, a powerful nostalgia for the time when history existed." Doesn't this vision seem exactly right? We appear to be losing a clear sense of both our history and our future, living in a perpetual present in which we have forgotten that things were different in the past and that there are, therefore, alternatives. (A parallel can perhaps be drawn with pop: we are in the post-postmodern age of the retro-authentic mashup. Contemporary songs - by Adele, Lady Gaga, La Roux - are simulacra of those produced in the 60s, 70s and 80s.)

I grew up in the 80s, marching against Thatcher. The left laid into the right. In 1990, when I turned 16, John Major became prime minister and the ideological clashes of British politics faded out. Major's "back to basics" campaign was against highfalutin ideology; a disavowal of politics. (In recent advice to Conservative MPs, Major told them to focus less on "ideology" and more on "issues that actually worry people in their daily lives". His rejection of the hardline right is to be applauded, but since when did daily issues have nothing to do with ideology?) Next came the triangulation of Tony Blair, his saintly transcendence of left and right; Barack Obama's call for "a declaration of independence … from ideology"; and David Cameron saying he "doesn't do isms". Politics is now a matter of technocratic optimisation, of doing "what works" and "getting the job done". In 2010, even the veteran conviction politician Shirley Williams praised the coalition government for its pledge to "work together in the national interest". "The generation I belong to, steeped in ideology and partisan commitment, is passing away," she wrote, commending a new spirit of "co-operation" over "the safe, long-established confrontation". While declaring that the old polarities no longer pertain, all the main parties have shifted to the right.

Meanwhile, the performance of confrontation continues. Popular disaffection with mainstream politics manifests as a rejection of its tribal, shouty style. PMQs is criticised for being too raucous, but that is a distracting irrelevance now that policy differences seem imperceptible. The problem is not "divisiveness" but its opposite: the lack of democratic choice.

In the recent commentary on the death of Tony Benn, he has been repeatedly described as one of the final representatives of a sharply delineated political culture. "To the modern eye he broke the mould: a brazen, aristocratic ideologue in an age of middle-class triangulation and third ways," wrote Mark Wallace, an editor at ConservativeHome. "But if those things seem so alien today, it's not because he was a one-off but because he was the last of his kind." The passing of political conviction is accepted as a given whatever one's political conviction, but it is the left that stands to lose most.

In The End of History and the Last Man, Fukuyama writes that the "enormously productive and dynamic economic world created by advancing technology" has a "tremendous homogenising power": global political harmony is the "ultimate victory of the VCR". But are consumerism and technology, as he suggests, really progressive? The internet came of age at the same time as I did. My undergraduate essays were handwritten, but in my third year I sent my first email using a green interface called Pine. My childhood correspondence fills several cardboard boxes, but during the 1990s the paper trail peters out. The rest is on email accounts owned by corporations with infantile names; some of those accounts are lost.

Is it an accident that the digital blitzing of boundaries between historical eras, work and play, this book and that, is happening at the same time as the seeming end of movements of all kinds, both cultural and political? My nostalgia for my own childhood is bound up with my nostalgia for political opposition and the material written word. I miss history, just as I miss my own history, and my childhood visions of the future. In my grander moments I feel like an embodiment of Slavoj Žižek's Living in the End Times, meandering mournfully around Spotify and fretting about the left's intellectual bankruptcy.

"The modern age was a time when human beings, alone or together, could sculpt the marble of history with the hammer of will," writes the writer and activist Franco "Bifo" Berardi. Today, this has "vanished from sight. There is no longer … a progressive temporal dimension."

As modernist housing projects fall into ruin, it is hard to recall the sincerity of Tomorrow's World or SF that is not ironically space age or steampunk. It is barely possible to articulate a utopia, even (or especially) if you are on the left. Is this because of neoliberalism's domination of the ideological landscape, or is it that we are in a post-ideological age, of which the internet is either a symptom or a cause? When every single person in a train carriage is staring at a small illuminated device, it is an almost tacky vision of dystopia. Technology – along with turbo-capitalism – seems to me to be hastening the cultural and environmental apocalypse. The way I see it, digital consumerism makes us too passive to revolt, or to save the world. If we accept it as inevitable it will indeed lead to the end of history, in more ways than one.

Is the recent challenging of Fukuyama's thesis grounds for new optimism? It is still too early to tell. "What is happening to us in the early years of the century," Badiou writes, is "something that would appear not to have any clear name in any accepted language." Fukuyama himself speculated that the absence of idealism and struggle might yet spark their rekindling: "Perhaps this very prospect of centuries of boredom at the end of history," he wrote, "will serve to get history started once again." There is a glimmer of perverse hope in the fact that boredom is a luxury most of us can no longer afford.

• Eliane Glaser's Get Real is published by Fourth Estate.