Last summer, researchers at the International Monetary Fund settled a long and bitter debate over “neoliberalism”: they admitted it exists. Three senior economists at the IMF, an organisation not known for its incaution, published a paper questioning the benefits of neoliberalism. In so doing, they helped put to rest the idea that the word is nothing more than a political slur, or a term without any analytic power. The paper gently called out a “neoliberal agenda” for pushing deregulation on economies around the world, for forcing open national markets to trade and capital, and for demanding that governments shrink themselves via austerity or privatisation. The authors cited statistical evidence for the spread of neoliberal policies since 1980, and their correlation with anaemic growth, boom-and-bust cycles and inequality.

Neoliberalism is an old term, dating back to the 1930s, but it has been revived as a way of describing our current politics – or more precisely, the range of thought allowed by our politics. In the aftermath of the 2008 financial crisis, it was a way of assigning responsibility for the debacle, not to a political party per se, but to an establishment that had conceded its authority to the market. For the Democrats in the US and Labour in the UK, this concession was depicted as a grotesque betrayal of principle. Bill Clinton and Tony Blair, it was said, had abandoned the left’s traditional commitments, especially to workers, in favour of a global financial elite and the self-serving policies that enriched them; and in doing so, had enabled a sickening rise in inequality.

Neoliberalism: the idea that swallowed the world – podcast Read more

Over the past few years, as debates have turned uglier, the word has become a rhetorical weapon, a way for anyone left of centre to incriminate those even an inch to their right. (No wonder centrists say it’s a meaningless insult: they’re the ones most meaningfully insulted by it.) But “neoliberalism” is more than a gratifyingly righteous jibe. It is also, in its way, a pair of eyeglasses.

Peer through the lens of neoliberalism and you see more clearly how the political thinkers most admired by Thatcher and Reagan helped shape the ideal of society as a kind of universal market (and not, for example, a polis, a civil sphere or a kind of family) and of human beings as profit-and-loss calculators (and not bearers of grace, or of inalienable rights and duties). Of course the goal was to weaken the welfare state and any commitment to full employment, and – always – to cut taxes and deregulate. But “neoliberalism” indicates something more than a standard rightwing wish list. It was a way of reordering social reality, and of rethinking our status as individuals.

Still peering through the lens, you see how, no less than the welfare state, the free market is a human invention. You see how pervasively we are now urged to think of ourselves as proprietors of our own talents and initiative, how glibly we are told to compete and adapt. You see the extent to which a language formerly confined to chalkboard simplifications describing commodity markets (competition, perfect information, rational behaviour) has been applied to all of society, until it has invaded the grit of our personal lives, and how the attitude of the salesman has become enmeshed in all modes of self-expression.

In short, “neoliberalism” is not simply a name for pro-market policies, or for the compromises with finance capitalism made by failing social democratic parties. It is a name for a premise that, quietly, has come to regulate all we practise and believe: that competition is the only legitimate organising principle for human activity.

No sooner had neoliberalism been certified as real, and no sooner had it made clear the universal hypocrisy of the market, than the populists and authoritarians came to power. In the US, Hillary Clinton, the neoliberal arch-villain, lost – and to a man who knew just enough to pretend he hated free trade. So are the eyeglasses now useless? Can they do anything to help us understand what is broken about British and American politics? Against the forces of global integration, national identity is being reasserted, and in the crudest possible terms. What could the militant parochialism of Brexit Britain and Trumpist America have to do with neoliberal rationality? What possible connection is there between the president – a freewheeling boob – and the bloodless paragon of efficiency known as the free market?

It isn’t only that the free market produces a tiny cadre of winners and an enormous army of losers – and the losers, looking for revenge, have turned to Brexit and Trump. There was, from the beginning, an inevitable relationship between the utopian ideal of the free market and the dystopian present in which we find ourselves; between the market as unique discloser of value and guardian of liberty, and our current descent into post-truth and illiberalism.

Moving the stale debate about neoliberalism forward begins, I think, with taking seriously the measure of its cumulative effect on all of us, regardless of affiliation. And this requires returning to its origins, which have nothing to do with Bill or Hillary Clinton. There once was a group of people who did call themselves neoliberals, and did so proudly, and their ambition was a total revolution in thought. The most prominent among them, Friedrich Hayek, did not think he was staking out a position on the political spectrum, or making excuses for the fatuous rich, or tinkering along the edges of microeconomics.

He thought he was solving the problem of modernity: the problem of objective knowledge. For Hayek, the market didn’t just facilitate trade in goods and services; it revealed truth. How did his ambition collapse into its opposite – the mind-bending possibility that, thanks to our thoughtless veneration of the free market, truth might be driven from public life altogether?

When the idea occurred to Friedrich Hayek in 1936, he knew, with the conviction of a “sudden illumination”, that he had struck upon something new. “How can the combination of fragments of knowledge existing in different minds,” he wrote, “bring about results which, if they were to be brought about deliberately, would require a knowledge on the part of the directing mind which no single person can possess?”

This was not a technical point about interest rates or deflationary slumps. This was not a reactionary polemic against collectivism or the welfare state. This was a way of birthing a new world. To his mounting excitement, Hayek understood that the market could be thought of as a kind of mind.

Adam Smith’s “invisible hand” had already given us the modern conception of the market: as an autonomous sphere of human activity and therefore, potentially, a valid object of scientific knowledge. But Smith was, until the end of his life, an 18th-century moralist. He thought the market could be justified only in light of individual virtue, and he was anxious that a society governed by nothing but transactional self-interest was no society at all. Neoliberalism is Adam Smith without the anxiety.

That Hayek is considered the grandfather of neoliberalism – a style of thought that reduces everything to economics – is a little ironic given that he was such a mediocre economist. He was just a young, obscure Viennese technocrat when he was recruited to the London School of Economics to compete with, or possibly even dim, the rising star of John Maynard Keynes at Cambridge.

The plan backfired, and Hayek lost out to Keynes in a rout. Keynes’s General Theory of Employment, Interest and Money, published in 1936, was greeted as a masterpiece. It dominated the public discussion, especially among young English economists in training, for whom the brilliant, dashing, socially connected Keynes was a beau idéal. By the end of the second world war, many prominent free-marketers had come around to Keynes’s way of thinking, conceding that government might play a role in managing a modern economy. The initial excitement over Hayek had dissipated. His peculiar notion that doing nothing could cure an economic depression had been discredited in theory and practice. He later admitted that he wished his work criticising Keynes would simply be forgotten.

Hayek cut a silly figure: a tall, erect, thickly accented professor in high-cut tweed, insisting on the formal “Von Hayek” but cruelly nicknamed “Mr Fluctooations” behind his back. In 1936, he was an academic without a portfolio and with no obvious future. Yet we now live in Hayek’s world, as we once lived in Keynes’s. Lawrence Summers, the Clinton adviser and former president of Harvard University, has said that Hayek’s conception of the price system as a mind is “as penetrating and original an idea as microeconomics produced in the 20th century” and “the single most important thing to learn from an economics course today”. This undersells it. Keynes did not make or predict the cold war, but his thinking wended its way into every aspect of the cold-war world; so too has Hayek’s thinking woven itself into every aspect of the post-1989 world.

Facebook Twitter Pinterest Friedrich Hayek teaching at the London School of Economics in 1948. Photograph: Paul Popper/Popperfoto/Getty

Hayek’s was a total worldview: a way of structuring all reality on the model of economic competition. He begins by assuming that nearly all (if not all) human activity is a form of economic calculation, and so can be assimilated to the master concepts of wealth, value, exchange, cost – and especially price. Prices are a means of allocating scarce resources efficiently, according to need and utility, as governed by supply and demand. For the price system to function efficiently, markets must be free and competitive. Ever since Smith imagined the economy as an autonomous sphere, the possibility existed that the market might not just be one piece of society, but society as a whole. Within such a society, men and women need only follow their own self-interest and compete for scarce rewards. Through competition, “it becomes possible”, as the sociologist Will Davies has written, “to discern who and what is valuable”.

What any person acquainted with history sees as the necessary bulwarks against tyranny and exploitation – a thriving middle class and civil sphere; free institutions; universal suffrage; freedom of conscience, congregation, religion and press; a basic recognition that the individual is a bearer of dignity – held no special place in Hayek’s thought. Hayek built into neoliberalism the assumption that the market provides all necessary protection against the one real political danger: totalitarianism. To prevent this, the state need only keep the market free.

This last is what makes neoliberalism “neo”. It is a crucial modification of the older belief in a free market and a minimal state, known as “classical liberalism”. In classical liberalism, merchants simply asked the state to “leave us alone” – to laissez-nous faire. Neoliberalism recognised that the state must be active in the organisation of a market economy. The conditions allowing for a free market must be won politically, and the state must be re-engineered to support the free market on an ongoing basis.

That isn’t all: every aspect of democratic politics, from the choices of voters to the decisions of politicians, must be submitted to a purely economic analysis. The lawmaker is obliged to leave well enough alone – to not distort the natural actions of the marketplace – and so, ideally, the state provides a fixed, neutral, universal legal framework within which market forces operate spontaneously. The conscious direction of government is never preferable to the “automatic mechanism of adjustment” – ie the price system, which is not only efficient but maximises liberty, or the opportunity for men and women to make free choices about their own lives.

As Keynes jetted between London and Washington, creating the postwar order, Hayek sat pouting in Cambridge. He had been sent there during the wartime evacuations; and he complained that he was surrounded by “foreigners” and “no lack of orientals of all kinds” and “Europeans of practically all nationalities, but very few of real intelligence”.

Stuck in England, without influence or respect, Hayek had only his idea to console him; an idea so grand it would one day dissolve the ground beneath the feet of Keynes and every other intellectual. Left to its own devices, the price system functions as a kind of mind. And not just any mind, but an omniscient one: the market computes what individuals cannot grasp. Reaching out to him as an intellectual comrade-in-arms, the American journalist Walter Lippmann wrote to Hayek, saying: “No human mind has ever understood the whole scheme of a society … At best a mind can understand its own version of the scheme, something much thinner, which bears to reality some such relation as a silhouette to a man.”

It is a grand epistemological claim – that the market is a way of knowing, one that radically exceeds the capacity of any individual mind. Such a market is less a human contrivance, to be manipulated like any other, than a force to be studied and placated. Economics ceases to be a technique – as Keynes believed it to be – for achieving desirable social ends, such as growth or stable money. The only social end is the maintenance of the market itself. In its omniscience, the market constitutes the only legitimate form of knowledge, next to which all other modes of reflection are partial, in both senses of the word: they comprehend only a fragment of a whole and they plead on behalf of a special interest. Individually, our values are personal ones, or mere opinions; collectively, the market converts them into prices, or objective facts.

After washing out at LSE, Hayek never held a permanent appointment that was not paid for by corporate sponsors. Even his conservative colleagues at the University of Chicago – the global epicentre of libertarian dissent in the 1950s – regarded Hayek as a reactionary mouthpiece, a “stock rightwing man” with a “stock rightwing sponsor”, as one put it. As late as 1972, a friend could visit Hayek, now in Salzburg, only to find an elderly man prostrate with self-pity, believing his life’s work was in vain. No one cared what he had written!

There had, however, been hopeful signs: Hayek was Barry Goldwater’s favourite political philosopher and was said to be Ronald Reagan’s, too. Then there was Margaret Thatcher. To anyone who would listen, Thatcher lionised Hayek, promising to bring together his free-market philosophy with a revival of Victorian values: family, community, hard work.

Hayek met privately with Thatcher in 1975, at the very moment that she, having been named leader of the opposition in the UK, was preparing to bring his Big Idea off the shelf and into history. They huddled for 30 minutes on Lord North Street in London, at the Institute for Economic Affairs. Afterwards, Thatcher’s staff anxiously asked Hayek what he had thought. What could he say? For the first time in 40 years, power was mirroring back to Friedrich von Hayek his own cherished self-image, a man who might vanquish Keynes and remake the world.

He replied: “She’s so beautiful.”

Hayek’s Big Idea isn’t much of an idea – until you supersize it. Organic, spontaneous, elegant processes that, like a million fingers on a Ouija board, coordinate to create outcomes that are otherwise unplanned. Applied to an actual market – one for pork bellies or corn futures – this description is little more than a truism. It can be expanded to describe how various markets, in commodities and labour and even money itself, form that part of a society known as “the economy”. This is less banal, but still inconsequential; a Keynesian accepts this description happily. But what if we bump it up one more step? What if we reconceive all of society as a kind of market?

The more Hayek’s idea expands, the more reactionary it gets, the more it hides behind its pretence of scientific neutrality – and the more it allows economics to link up with the major intellectual trend of the west since the 17th century. The rise of modern science generated a problem: if the world is universally obedient to natural laws, what does it mean to be human? Is a human being simply an object in the world, like any other? There appears to be no way to assimilate the subjective, interior human experience into nature as science conceives it – as something objective whose rules we discover by observation.

Everything about the postwar political culture lay in favour of John Maynard Keynes, and an expanded role for the state in managing the economy. But everything about the postwar academic culture lay in favour of Hayek’s Big Idea. Before the war, even the most rightwing economist thought of the market as a means to a limited end, to the efficient allocation of scarce resources. From the time of Adam Smith in the mid-1700s, and up to that of the founding members of the Chicago school in the postwar years, it was commonplace to believe that the ultimate ends of society and of life, were established in the non-economic sphere.

Facebook Twitter Pinterest John Maynard Keynes, circa 1940. Photograph: Tim Gidal/Getty

On this view, questions of value are resolved politically and democratically, not economically – through moral reflection and public deliberation. The classic modern expression of this belief is found in a 1922 essay called Ethics and the Economic Interpretation by Frank Knight, who arrived at Chicago two decades before Hayek. “The rational economic criticism of values gives results repugnant to common sense,” Knight wrote. “Economic man is the selfish, ruthless object of moral condemnation.”

Economists had struggled for 200 years with the question of how to place the values on which an otherwise commercial society is organised beyond mere self-interest and calculation. Knight, along with his colleagues Henry Simons and Jacob Viner, were holdouts against Franklin D Roosevelt and the market interventions of the New Deal, and they established the University of Chicago as the intellectually rigorous home of free-market economics that it remains to this day. However, Simons, Viner and Knight all started their careers before the unrivalled prestige of atomic physicists drew enormous sums of money into the university system and kicked off a postwar vogue for “hard” science. They did not worship equations or models, and they worried about non-scientific questions. Most explicitly, they worried about questions of value, where value was absolutely distinct from price.

It is not just that Simons, Viner and Knight were less dogmatic than Hayek, or more willing to pardon the state for taxing and spending. It is not the case that Hayek was their intellectual superior. But they acknowledged as a first principle that society was not the same thing as the market, and that price was not the same thing as value. This set them up to be swallowed whole by history.

It was Hayek who showed us how to get from the hopeless condition of human partiality to the majestic objectivity of science. Hayek’s Big Idea acts as the missing link between our subjective human nature, and nature itself. In so doing, it puts any value that cannot be expressed as a price – as the verdict of a market – on an equally unsure footing, as nothing more than opinion, preference, folklore or superstition.

More than anyone, even Hayek himself, it was the great postwar Chicago economist Milton Friedman who helped convert governments and politicians to the power of Hayek’s Big Idea. But first he broke with two centuries of precedent and declared that economics is “in principle independent of any particular ethical position or normative judgments” and is “an ‘objective’ science, in precisely the same sense as any of the physical sciences”. Values of the old, mental, normative kind were defective, they were “differences about which men can ultimately only fight”. There is the market, in other words, and there is relativism.

Markets may be human facsimiles of natural systems, and like the universe itself, they may be authorless and valueless. But the application of Hayek’s Big Idea to every aspect of our lives negates what is most distinctive about us. That is, it assigns what is most human about human beings – our minds and our volition – to algorithms and markets, leaving us to mimic, zombie-like, the shrunken idealisations of economic models. Supersizing Hayek’s idea and radically upgrading the price system into a kind of social omniscience means radically downgrading the importance of our individual capacity to reason – our ability to provide and evaluate justifications for our actions and beliefs.

As a result, the public sphere – the space where we offer up reasons, and contest the reasons of others – ceases to be a space for deliberation, and becomes a market in clicks, likes and retweets. The internet is personal preference magnified by algorithm; a pseudo-public space that echoes the voice already inside our head. Rather than a space of debate in which we make our way, as a society, toward consensus, now there is a mutual-affirmation apparatus banally referred to as a “marketplace of ideas”. What looks like something public and lucid is only an extension of our own pre-existing opinions, prejudices and beliefs, while the authority of institutions and experts has been displaced by the aggregative logic of big data. When we access the world through a search engine, its results are ranked, as the founder of Google puts it, “recursively” – by an infinity of individual users functioning as a market, continuously and in real time.

The awesome utilities of digital technology aside, an earlier and more humanist tradition, which was dominant for centuries, had always distinguished between our tastes and preferences – the desires that find expression in the market – and our capacity for reflection on those preferences, which allows us to form and express values.

“A taste is almost defined as a preference about which you do not argue,” the philosopher and economist Albert O Hirschman once wrote. “A taste about which you argue, with others or yourself, ceases ipso facto being a taste – it turns into a value.”

Hirschman drew a distinction between that part of one’s self that is a consumer, and that part of one’s self that is a supplier of reasons. The market reflects what Hirschman called the preferences that are “revealed by agents as they buy goods and services”. But, as he puts it, men and women also “have the ability to step back from their ‘revealed’ wants, volition and preferences, to ask themselves whether they really want these wants and prefer these preferences”. We fashion our selves and identities on the basis of this capacity for reflection. The use of one’s individual reflective powers is reason; the collective use of these reflective powers is public reason; the use of public reason to make law and policy is democracy. When we provide reasons for our actions and beliefs, we bring ourselves into being: individually and collectively, we decide who and what we are.

According to the logic of Hayek’s Big Idea, these expressions of human subjectivity are meaningless without ratification by the market – as Friedman said, they are nothing but relativism, each as good as any other. When the only objective truth is determined by the market, all other values have the status of mere opinions; everything else is relativist hot air. But Friedman’s “relativism” is a charge that can be thrown at any claim based on human reason. It is a nonsense insult, as all humanistic pursuits are “relative” in a way the sciences are not. They are relative to the (private) condition of having a mind, and the (public) need to reason and understand even when we can’t expect scientific proof. When our debates are no longer resolved by deliberation over reasons, then the whimsies of power will determine the outcome.

This is where the triumph of neoliberalism meets the political nightmare we are living through now. “You had one job,” the old joke goes, and Hayek’s grand project, as originally conceived in 30s and 40s, was explicitly designed to prevent a backslide into political chaos and fascism. But the Big Idea was always this abomination waiting to happen. It was, from the beginning, pregnant with the thing it was said to protect against. Society reconceived as a giant market leads to a public life lost to bickering over mere opinions; until the public turns, finally, in frustration to a strongman as a last resort for solving its otherwise intractable problems.

In 1989, an American reporter knocked on the 90-year-old Hayek’s door. He was living in Freiburg, West Germany, in a third-floor apartment in a stucco house on Urachstrasse. The two men sat in a sunroom whose windows looked out on the mountains, and Hayek, who was recovering from pneumonia, pulled a blanket over his legs as they spoke.

This was no longer the man who had once wallowed in his own defeat at the hands of Keynes. Thatcher had just written to Hayek in a tone of millennial triumph. None of what she and Reagan had accomplished “would have been possible without the values and beliefs to set us on the right road and provide the right sense of direction”. Hayek was now cheerful on his own account, and optimistic about the future of capitalism. As the journalist wrote, “In particular, Hayek sees a greater appreciation for the market among the younger generation. Today unemployed youth in Algiers and Rangoon riot not for centrally planned welfare state but for opportunity: the freedom to buy and sell – jeans, cars, whatever – at whatever prices the market will bear.”

Thirty years on, and it can fairly be said that Hayek’s victory is unrivalled. We live in a paradise built by his Big Idea. The more closely the world can be made to resemble an ideal market governed only by perfect competition, the more law-like and “scientific” human behaviour, in the aggregate, becomes. Every day we ourselves – no one has to tell us to anymore! – strive to become more perfectly like scattered, discrete, anonymous buyers and sellers; and every day we treat the residual desire to be something more than a consumer as nostalgia, or elitism.

How statistics lost their power – and why we should fear what comes next | William Davies Read more

What began as a new form of intellectual authority, rooted in a devoutly apolitical worldview, nudged easily into an ultra-reactionary politics. What can’t be quantified must not be real, says the economist, and how do you measure the benefits of the core faiths of the enlightenment – namely, critical reasoning, personal autonomy and democratic self-government? When we abandoned, for its embarrassing residue of subjectivity, reason as a form of truth, and made science the sole arbiter of both the real and the true, we created a void that pseudo-science was happy to fill.

The authority of the professor, the reformer, the legislator or the jurist does not derive from the market, but from humanistic values such as public spiritedness, conscience or the longing for justice. Long before the Trump administration started demeaning them, such figures had been drained of salience by an explanatory scheme that can’t explain them. Surely there is a connection between their growing irrelevance and the election of Trump, a creature of pure whim, a man without the principles or conviction to make for a coherent self. A man without a mind, who represents the total absence of reason, is running the world; or at least ruining it. As a Manhattan real estate wiseguy, though, Trump, hey – he knows what he knows: that his sins have yet to be punished in the marketplace.

Main illustration by Bratislav Milenkovic

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.