For a manifesto to succeed, it must speak to our hearts like a poem while infecting the mind with images and ideas that are dazzlingly new. It needs to open our eyes to the true causes of the bewildering, disturbing, exciting changes occurring around us, exposing the possibilities with which our current reality is pregnant. It should make us feel hopelessly inadequate for not having recognised these truths ourselves, and it must lift the curtain on the unsettling realisation that we have been acting as petty accomplices, reproducing a dead-end past. Lastly, it needs to have the power of a Beethoven symphony, urging us to become agents of a future that ends unnecessary mass suffering and to inspire humanity to realise its potential for authentic freedom.

Yanis Varoufakis: Marx predicted our present crisis, and points the way out – podcast Read more

No manifesto has better succeeded in doing all this than the one published in February 1848 at 46 Liverpool Street, London. Commissioned by English revolutionaries, The Communist Manifesto (or the Manifesto of the Communist Party, as it was first published) was authored by two young Germans – Karl Marx, a 29-year-old philosopher with a taste for epicurean hedonism and Hegelian rationality, and Friedrich Engels, a 28-year-old heir to a Manchester mill.

As a work of political literature, the manifesto remains unsurpassed. Its most infamous lines, including the opening one (“A spectre is haunting Europe – the spectre of communism”), have a Shakespearean quality. Like Hamlet confronted by the ghost of his slain father, the reader is compelled to wonder: “Should I conform to the prevailing order, suffering the slings and arrows of the outrageous fortune bestowed upon me by history’s irresistible forces? Or should I join these forces, taking up arms against the status quo and, by opposing it, usher in a brave new world?”

For Marx and Engels’ immediate readership, this was not an academic dilemma, debated in the salons of Europe. Their manifesto was a call to action, and heeding this spectre’s invocation often meant persecution, or, in some cases, lengthy imprisonment. Today, a similar dilemma faces young people: conform to an established order that is crumbling and incapable of reproducing itself, or oppose it, at considerable personal cost, in search of new ways of working, playing and living together? Even though communist parties have disappeared almost entirely from the political scene, the spirit of communism driving the manifesto is proving hard to silence.

To see beyond the horizon is any manifesto’s ambition. But to succeed as Marx and Engels did in accurately describing an era that would arrive a century-and-a-half in the future, as well as to analyse the contradictions and choices we face today, is truly astounding. In the late 1840s, capitalism was foundering, local, fragmented and timid. And yet Marx and Engels took one long look at it and foresaw our globalised, financialised, iron-clad, all-singing-all-dancing capitalism. This was the creature that came into being after 1991, at the very same moment the establishment was proclaiming the death of Marxism and the end of history.

Of course, the predictive failure of The Communist Manifesto has long been exaggerated. I remember how even leftwing economists in the early 1970s challenged the pivotal manifesto prediction that capital would “nestle everywhere, settle everywhere, establish connexions everywhere”. Drawing upon the sad reality of what were then called third world countries, they argued that capital had lost its fizz well before expanding beyond its “metropolis” in Europe, America and Japan.

Empirically they were correct: European, US and Japanese multinational corporations operating in the “peripheries” of Africa, Asia and Latin America were confining themselves to the role of colonial resource extractors and failing to spread capitalism there. Instead of imbuing these countries with capitalist development (drawing “all, even the most barbarian, nations into civilisation”), they argued that foreign capital was reproducing the development of underdevelopment in the third world. It was as if the manifesto had placed too much faith in capital’s ability to spread into every nook and cranny. Most economists, including those sympathetic to Marx, doubted the manifesto’s prediction that “exploitation of the world-market” would give “a cosmopolitan character to production and consumption in every country”.

As it turned out, the manifesto was right, albeit belatedly. It would take the collapse of the Soviet Union and the insertion of two billion Chinese and Indian workers into the capitalist labour market for its prediction to be vindicated. Indeed, for capital to globalise fully, the regimes that pledged allegiance to the manifesto had first to be torn asunder. Has history ever procured a more delicious irony?

Anyone reading the manifesto today will be surprised to discover a picture of a world much like our own, teetering fearfully on the edge of technological innovation. In the manifesto’s time, it was the steam engine that posed the greatest challenge to the rhythms and routines of feudal life. The peasantry were swept into the cogs and wheels of this machinery and a new class of masters, the factory owners and the merchants, usurped the landed gentry’s control over society. Now, it is artificial intelligence and automation that loom as disruptive threats, promising to sweep away “all fixed, fast-frozen relations”. “Constantly revolutionising … instruments of production,” the manifesto proclaims, transform “the whole relations of society”, bringing about “constant revolutionising of production, uninterrupted disturbance of all social conditions, everlasting uncertainty and agitation”.

Facebook Twitter Pinterest Composite: Guardian Design

For Marx and Engels, however, this disruption is to be celebrated. It acts as a catalyst for the final push humanity needs to do away with our remaining prejudices that underpin the great divide between those who own the machines and those who design, operate and work with them. “All that is solid melts into air, all that is holy is profaned,” they write in the manifesto of technology’s effect, “and man is at last compelled to face with sober senses, his real conditions of life, and his relations with his kind”. By ruthlessly vaporising our preconceptions and false certainties, technological change is forcing us, kicking and screaming, to face up to how pathetic our relations with one another are.

Today, we see this reckoning in millions of words, in print and online, used to debate globalisation’s discontents. While celebrating how globalisation has shifted billions from abject poverty to relative poverty, venerable western newspapers, Hollywood personalities, Silicon Valley entrepreneurs, bishops and even multibillionaire financiers all lament some of its less desirable ramifications: unbearable inequality, brazen greed, climate change, and the hijacking of our parliamentary democracies by bankers and the ultra-rich.

None of this should surprise a reader of the manifesto. “Society as a whole,” it argues, “is more and more splitting up into two great hostile camps, into two great classes directly facing each other.” As production is mechanised, and the profit margin of the machine-owners becomes our civilisation’s driving motive, society splits between non-working shareholders and non-owner wage-workers. As for the middle class, it is the dinosaur in the room, set for extinction.

At the same time, the ultra-rich become guilt-ridden and stressed as they watch everyone else’s lives sink into the precariousness of insecure wage-slavery. Marx and Engels foresaw that this supremely powerful minority would eventually prove “unfit to rule” over such polarised societies, because they would not be in a position to guarantee the wage-slaves a reliable existence. Barricaded in their gated communities, they find themselves consumed by anxiety and incapable of enjoying their riches. Some of them, those smart enough to realise their true long-term self-interest, recognise the welfare state as the best available insurance policy. But alas, explains the manifesto, as a social class, it will be in their nature to skimp on the insurance premium, and they will work tirelessly to avoid paying the requisite taxes.

Is this not what has transpired? The ultra-rich are an insecure, permanently disgruntled clique, constantly in and out of detox clinics, relentlessly seeking solace from psychics, shrinks and entrepreneurial gurus. Meanwhile, everyone else struggles to put food on the table, pay tuition fees, juggle one credit card for another or fight depression. We act as if our lives are carefree, claiming to like what we do and do what we like. Yet in reality, we cry ourselves to sleep.

Do-gooders, establishment politicians and recovering academic economists all respond to this predicament in the same way, issuing fiery condemnations of the symptoms (income inequality) while ignoring the causes (exploitation resulting from the unequal property rights over machines, land, resources). Is it any wonder we are at an impasse, wallowing in hopelessness that only serves the populists seeking to court the worst instincts of the masses?

With the rapid rise of advanced technology, we are brought closer to the moment when we must decide how to relate to each other in a rational, civilised manner. We can no longer hide behind the inevitability of work and the oppressive social norms it necessitates. The manifesto gives its 21st-century reader an opportunity to see through this mess and to recognise what needs to be done so that the majority can escape from discontent into new social arrangements in which “the free development of each is the condition for the free development of all”. Even though it contains no roadmap of how to get there, the manifesto remains a source of hope not to be dismissed.

If the manifesto holds the same power to excite, enthuse and shame us that it did in 1848, it is because the struggle between social classes is as old as time itself. Marx and Engels summed this up in 13 audacious words: “The history of all hitherto existing society is the history of class struggles.”

From feudal aristocracies to industrialised empires, the engine of history has always been the conflict between constantly revolutionising technologies and prevailing class conventions. With each disruption of society’s technology, the conflict between us changes form. Old classes die out and eventually only two remain standing: the class that owns everything and the class that owns nothing – the bourgeoisie and the proletariat.

This is the predicament in which we find ourselves today. While we owe capitalism for having reduced all class distinctions to the gulf between owners and non-owners, Marx and Engels want us to realise that capitalism is insufficiently evolved to survive the technologies it spawns. It is our duty to tear away at the old notion of privately owned means of production and force a metamorphosis, which must involve the social ownership of machinery, land and resources. Now, when new technologies are unleashed in societies bound by the primitive labour contract, wholesale misery follows. In the manifesto’s unforgettable words: “A society that has conjured up such gigantic means of production and of exchange, is like the sorcerer who is no longer able to control the powers of the nether world whom he has called up by his spells.”

The sorcerer will always imagine that their apps, search engines, robots and genetically engineered seeds will bring wealth and happiness to all. But, once released into societies divided between wage labourers and owners, these technological marvels will push wages and prices to levels that create low profits for most businesses. It is only big tech, big pharma and the few corporations that command exceptionally large political and economic power over us that truly benefit. If we continue to subscribe to labour contracts between employer and employee, then private property rights will govern and drive capital to inhuman ends. Only by abolishing private ownership of the instruments of mass production and replacing it with a new type of common ownership that works in sync with new technologies, will we lessen inequality and find collective happiness.

According to Marx and Engels’ 13-word theory of history, the current stand-off between worker and owner has always been guaranteed. “Equally inevitable,” the manifesto states, is the bourgeoisie’s “fall and the victory of the proletariat”. So far, history has not fulfilled this prediction, but critics forget that the manifesto, like any worthy piece of propaganda, presents hope in the form of certainty. Just as Lord Nelson rallied his troops before the Battle of Trafalgar by announcing that England “expected” them to do their duty (even if he had grave doubts that they would), the manifesto bestows upon the proletariat the expectation that they will do their duty to themselves, inspiring them to unite and liberate one another from the bonds of wage-slavery.

Will they? On current form, it seems unlikely. But, then again, we had to wait for globalisation to appear in the 1990s before the manifesto’s estimation of capital’s potential could be fully vindicated. Might it not be that the new global, increasingly precarious proletariat needs more time before it can play the historic role the manifesto anticipated? While the jury is still out, Marx and Engels tell us that, if we fear the rhetoric of revolution, or try to distract ourselves from our duty to one another, we will find ourselves caught in a vertiginous spiral in which capital saturates and bleaches the human spirit. The only thing we can be certain of, according to the manifesto, is that unless capital is socialised we are in for dystopic developments.

On the topic of dystopia, the sceptical reader will perk up: what of the manifesto’s own complicity in legitimising authoritarian regimes and steeling the spirit of gulag guards? Instead of responding defensively, pointing out that no one blames Adam Smith for the excesses of Wall Street, or the New Testament for the Spanish Inquisition, we can speculate how the authors of the manifesto might have answered this charge. I believe that, with the benefit of hindsight, Marx and Engels would confess to an important error in their analysis: insufficient reflexivity. This is to say that they failed to give sufficient thought, and kept a judicious silence, over the impact their own analysis would have on the world they were analysing.

The manifesto told a powerful story in uncompromising language, intended to stir readers from their apathy. What Marx and Engels failed to foresee was that powerful, prescriptive texts have a tendency to procure disciples, believers – a priesthood, even – and that this faithful might use the power bestowed upon them by the manifesto to their own advantage. With it, they might abuse other comrades, build their own power base, gain positions of influence, bed impressionable students, take control of the politburo and imprison anyone who resists them.

Similarly, Marx and Engels failed to estimate the impact of their writing on capitalism itself. To the extent that the manifesto helped fashion the Soviet Union, its eastern European satellites, Castro’s Cuba, Tito’s Yugoslavia and several social democratic governments in the west, would these developments not cause a chain reaction that would frustrate the manifesto’s predictions and analysis? After the Russian revolution and then the second world war, the fear of communism forced capitalist regimes to embrace pension schemes, national health services, even the idea of making the rich pay for poor and petit bourgeois students to attend purpose-built liberal universities. Meanwhile, rabid hostility to the Soviet Union stirred up paranoia and created a climate of fear that proved particularly fertile for figures such as Joseph Stalin and Pol Pot.

I believe that Marx and Engels would have regretted not anticipating the manifesto’s impact on the communist parties it foreshadowed. They would be kicking themselves that they overlooked the kind of dialectic they loved to analyse: how workers’ states would become increasingly totalitarian in their response to capitalist state aggression, and how, in their response to the fear of communism, these capitalist states would grow increasingly civilised.

Blessed, of course, are the authors whose errors result from the power of their words. Even more blessed are those whose errors are self-correcting. In our present day, the workers’ states inspired by the manifesto are almost gone, and the communist parties disbanded or in disarray. Liberated from competition with regimes inspired by the manifesto, globalised capitalism is behaving as if it is determined to create a world best explained by the manifesto.

What makes the manifesto truly inspiring today is its recommendation for us in the here and now, in a world where our lives are being constantly shaped by what Marx described in his earlier Economic and Philosophical Manuscripts as “a universal energy which breaks every limit and every bond and posits itself as the only policy, the only universality, the only limit and the only bond”. From Uber drivers and finance ministers to banking executives and the wretchedly poor, we can all be excused for feeling overwhelmed by this “energy”. Capitalism’s reach is so pervasive it can sometimes seem impossible to imagine a world without it. It is only a small step from feelings of impotence to falling victim to the assertion there is no alternative. But, astonishingly (claims the manifesto), it is precisely when we are about to succumb to this idea that alternatives abound.

What we don’t need at this juncture are sermons on the injustice of it all, denunciations of rising inequality or vigils for our vanishing democratic sovereignty. Nor should we stomach desperate acts of regressive escapism: the cry to return to some pre-modern, pre-technological state where we can cling to the bosom of nationalism. What the manifesto promotes in moments of doubt and submission is a clear-headed, objective assessment of capitalism and its ills, seen through the cold, hard light of rationality.

Facebook Twitter Pinterest Composite: Guardian Design

The manifesto argues that the problem with capitalism is not that it produces too much technology, or that it is unfair. Capitalism’s problem is that it is irrational. Capital’s success at spreading its reach via accumulation for accumulation’s sake is causing human workers to work like machines for a pittance, while the robots are programmed to produce stuff that the workers can no longer afford and the robots do not need. Capital fails to make rational use of the brilliant machines it engenders, condemning whole generations to deprivation, a decrepit environment, underemployment and zero real leisure from the pursuit of employment and general survival. Even capitalists are turned into angst-ridden automatons. They live in permanent fear that unless they commodify their fellow humans, they will cease to be capitalists – joining the desolate ranks of the expanding precariat-proletariat.

If capitalism appears unjust it is because it enslaves everyone, rich and poor, wasting human and natural resources. The same “production line” that pumps out untold wealth also produces deep unhappiness and discontent on an industrial scale. So, our first task – according to the manifesto – is to recognise the tendency of this all-conquering “energy” to undermine itself.

When asked by journalists who or what is the greatest threat to capitalism today, I defy their expectations by answering: capital! Of course, this is an idea I have been plagiarising for decades from the manifesto. Given that it is neither possible nor desirable to annul capitalism’s “energy”, the trick is to help speed up capital’s development (so that it burns up like a meteor rushing through the atmosphere) while, on the other hand, resisting (through rational, collective action) its tendency to steamroller our human spirit. In short, the manifesto’s recommendation is that we push capital to its limits while limiting its consequences and preparing for its socialisation.

We need more robots, better solar panels, instant communication and sophisticated green transport networks. But equally, we need to organise politically to defend the weak, empower the many and prepare the ground for reversing the absurdities of capitalism. In practical terms, this means treating the idea that there is no alternative with the contempt it deserves while rejecting all calls for a “return” to a less modernised existence. There was nothing ethical about life under earlier forms of capitalism. TV shows that massively invest in calculated nostalgia, such as Downton Abbey, should make us glad to live when we do. At the same time, they might also encourage us to floor the accelerator of change.

The manifesto is one of those emotive texts that speak to each of us differently at different times, reflecting our own circumstances. Some years ago, I called myself an erratic, libertarian Marxist and I was roundly disparaged by non-Marxists and Marxists alike. Soon after, I found myself thrust into a political position of some prominence, during a period of intense conflict between the then Greek government and some of capitalism’s most powerful agents. Rereading the manifesto for the purposes of writing this introduction has been a little like inviting the ghosts of Marx and Engels to yell a mixture of censure and support in my ear.

Adults in the Room, my memoir of the time I served as Greece’s finance minister in 2015, tells the story of how the Greek spring was crushed via a combination of brute force (on the part of Greece’s creditors) and a divided front within my own government. It is as honest and accurate as I could make it. Seen from the perspective of the manifesto, however, the true historical agents were confined to cameo appearances or to the role of quasi-passive victims. “Where is the proletariat in your story?” I can almost hear Marx and Engels screaming at me now. “Should they not be the ones confronting capitalism’s most powerful, with you supporting from the sidelines?”

Yanis Varoufakis: How I became an erratic Marxist Read more

Thankfully, rereading the manifesto has offered some solace too, endorsing my view of it as a liberal text – a libertarian one, even. Where the manifesto lambasts bourgeois-liberal virtues, it does so because of its dedication and even love for them. Liberty, happiness, autonomy, individuality, spirituality, and self-guided development are ideals that Marx and Engels valued above everything else. If they are angry with the bourgeoisie, it is because the bourgeoisie seeks to deny the majority any opportunity to be free. Given Marx and Engels’ adherence to Hegel’s fantastic idea that no one is free as long as one person is in chains, their quarrel with the bourgeoisie is that they sacrifice everybody’s freedom and individuality on capitalism’s altar of accumulation.

Although Marx and Engels were not anarchists, they loathed the state and its potential to be manipulated by one class to suppress another. At best, they saw it as a necessary evil that would live on in the good, post-capitalist future coordinating a classless society. If this reading of the manifesto holds water, the only way of being a communist is to be a libertarian one. Heeding the manifesto’s call to “Unite!” is in fact inconsistent with becoming card-carrying Stalinists or with seeking to remake the world in the image of now-defunct communist regimes.

When everything is said and done, then, what is the bottom line of the manifesto? And why should anyone, especially young people today, care about history, politics and the like?

Marx and Engels based their manifesto on a touchingly simple answer: authentic human happiness and the genuine freedom that must accompany it. For them, these are the only things that truly matter. Their manifesto does not rely on strict Germanic invocations of duty, or appeals to historic responsibilities to inspire us to act. It does not moralise, or point its finger. Marx and Engels attempted to overcome the fixations of German moral philosophy and capitalist profit motives, with a rational, yet rousing appeal to the very basics of our shared human nature.

Key to their analysis is the ever-expanding chasm between those who produce and those who own the instruments of production. The problematic nexus of capital and waged labour stops us from enjoying our work and our artefacts, and turns employers and workers, rich and poor, into mindless, quivering pawns who are being quick-marched towards a pointless existence by forces beyond our control.

But why do we need politics to deal with this? Isn’t politics stultifying, especially socialist politics, which Oscar Wilde once claimed “takes up too many evenings”? Marx and Engels’ answer is: because we cannot end this idiocy individually; because no market can ever emerge that will produce an antidote to this stupidity. Collective, democratic political action is our only chance for freedom and enjoyment. And for this, the long nights seem a small price to pay.

Humanity may succeed in securing social arrangements that allow for “the free development of each” as the “condition for the free development of all”. But, then again, we may end up in the “common ruin” of nuclear war, environmental disaster or agonising discontent. In our present moment, there are no guarantees. We can turn to the manifesto for inspiration, wisdom and energy but, in the end, what prevails is up to us.

Adapted from Yanis Varoufakis’s introduction to The Communist Manifesto, published by Vintage Classics on 26 April

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.