On a late Friday afternoon in November last year, police were called to London’s Oxford Circus for reasons described as “terror-related”. Oxford Circus underground station was evacuated, producing a crush of people as they made for the exits. Reports circulated of shots being fired, and photos and video appeared online of crowds fleeing the area, with heavily armed police officers heading in the opposite direction. Amid the panic, it was unclear where exactly the threat was emanating from, or whether there might be a number of attacks going on simultaneously, as had occurred in Paris two years earlier. Armed police stormed Selfridges department store, while shoppers were instructed to evacuate the building. Inside the shop at the time was the pop star Olly Murs, who tweeted to nearly 8 million followers: “Fuck everyone get out of Selfridge now gun shots!!” As shoppers in the store made for the exits, others were rushing in at the same time, producing a stampede.

Smartphones and social media meant that this whole event was recorded, shared and discussed in real time. The police attempted to quell the panic using their own Twitter feed, but this was more than offset by the sense of alarm that was engulfing other observers. Far-right campaigner Tommy Robinson tweeted that this “looks like another jihad attack in London”. The Daily Mail unearthed an innocent tweet from 10 days earlier, which had described a “lorry stopped on a pavement in Oxford Street”, and used this as a basis on which to tweet “Gunshots fired” as armed police officers surrounded Oxford Circus station after “lorry ploughs into pedestrians”. The media were not so much reporting facts, as serving to synchronise attention and emotion across a watching public.

Around an hour after the initial evacuation of Oxford Circus, the police put out a statement that “to date police have not located any trace of any suspects, evidence of shots fired or casualties”. It subsequently emerged that nine people required treatment in hospital for injuries sustained in the panic, but nothing more serious had yet been discovered. A few minutes later, the London Underground authority tweeted that stations had reopened and trains were running normally. There were no guns and no terrorists.

What had caused this event? The police had received numerous calls from members of the public reporting gunshots on the underground and at street level, and had arrived within six minutes ready to respond. But the only violence that anyone had witnessed with their eyes was a scuffle on an overcrowded rush-hour platform, as two men bumped into each other, and a punch or two was thrown. While it remained unclear what had caused the impression of shots being fired, the scuffle had been enough to lead the surrounding crowd to retreat suddenly in fear, producing a wave of rapid movement that was then amplified as it spread along the busy platform and through the station. Given that there had been terrorist attacks in London earlier in the year and others reportedly foiled by the police, it is not hard to understand how panic might have spread in such confined spaces. Nobody would expect people to act in accordance with the facts in the heat of the moment, as a mass of bodies are hurtling and screaming around them. Where rapid response is essential, bodily instinct takes hold.

False alarm … police officers near Selfridges on Oxford Street, London. Photograph: Simon Dawson/Reuters

Following the Oxford Circus incident, local shopkeepers demanded the installation of a “Tokyo-style” loudspeaker system in the surrounding streets to allow the police to communicate with entire crowds all at once. The idea gained little traction but did partly diagnose the problem. Where events are unfolding rapidly and emotions are riding high, there is a sudden absence of any authoritative perspective on reality. In the digital age, that vacuum of hard knowledge becomes rapidly filled by rumours, fantasy and guesswork, some of which is quickly twisted and exaggerated to suit a preferred narrative. Fear of violence can be just as disruptive a force as actual violence, and it can be difficult to quell once it is at large.

Given sufficient speed, virality can generate fear and conflict out of thin air. Research recently published by Warwick University showed a correlation between anti-refugee hate crimes in Germany and levels of Facebook use, indicating the danger and paranoia that rapid peer-to-peer information transmission can manufacture. Facebook has also been implicated in ethnic violence in Sri Lanka, Myanmar and India, due to online rumours triggering physical mobilisations and confrontations.

We can confirm that Selfridges London has been evacuated as a precautionary measure. We can confirm that there have been no reported incidents in store — Selfridges (@Selfridges) November 24, 2017

Events such as these distil something about the times in which we live, when speed of reaction often takes precedence over slower and more cautious assessments. As we become more attuned to “real time” media, we inevitably end up placing more trust in sensation and emotion than in evidence. Knowledge becomes more valued for its speed and impact than for its cold objectivity, and – as studies of Twitter content have confirmed – emotive falsehood often travels faster than fact. In situations of physical danger, where time is of the essence, rapid reaction makes sense. But the influence of “real time” data now extends well beyond matters of security. News, financial markets, friendships and work engage us in a constant flow of information, making it harder to stand back and construct a more reliable or consensual portrait of any of them. The threat lurking in this is that otherwise peaceful situations can come to feel dangerous, until eventually they really are.

Telling people to keep emotions in check and bow down before their more rational superiors isn't an effective message

In the context of rising populism, and the accompanying syndromes denounced as “post-truth” and “tribalism”, it is common to blame political turbulence on voters giving vent to their feelings. But telling people to keep their emotions in check, and bow down before their more rational superiors, is scarcely an appealing or effective message. To criticise populists and their supporters in these terms is to avoid confronting the much larger question of why feeling might have become a more important navigational aid in the 21st century. Rather than criticise people for a lack of self-control, how might we understand the historical transition that has turned feeling into such a potent political force?

This question returns us to some foundational philosophical issues. The modern world was founded on two fundamental distinctions, both inaugurated in the mid-17th century: between mind and body, and between war and peace. These binaries have been gradually weakening for more than 100 years. The rise of psychology and psychiatry in the late 19th century brought mind and body into closer proximity to each other, demonstrating how our thoughts are influenced by nervous impulses and feelings. The invention of aerial bombing in the early 20th century meant that war came to include techniques for terrifying and policing civilian populations.

These two distinctions – between mind and body, and war and peace – now appear to have lost credibility altogether, with the result that we experience conflict intruding into everyday life with increasing regularity. Since the 1990s, rapid advances in neuroscience have elevated the brain over the mind as the main way by which we understand ourselves, demonstrating the importance of emotion and physiology to all decision making. Meanwhile, new forms of violence have emerged, in which states are attacked by non-state groups (such as Islamic State), interstate conflicts are fought using nonmilitary means (such as cyberwarfare), and the distinction between policing and military intervention becomes blurred. Our condition is one of nervous states, with individuals and governments existing in a state of constant and heightened alertness, relying increasingly on feeling rather than fact.

Red alert … Illustration by Andrea Ucini

When we speak of feeling something, this can mean two different things. First, there is physical sensation, including pleasure and pain, which is crucial for navigating our environment. Our nervous system receives sensations from the outside world that are used to coordinate our bodies and instinctive movements. The brilliance of our neurological network is that it facilitates immediate response to new information, whether that be from our physical circumstances or our internal organs. The brain manages sensory impressions extremely rapidly, offering among other things a crucial defence against external threats. Individual sensations may not count as knowledge or facts, but they are an indispensable form of data, which we rely on almost constantly.

Second, there are feelings in the sense of emotions. These are experiences that we are capable of consciously reflecting on and articulating. We have a wide vocabulary for naming and expressing these feelings. We communicate them physically in our facial expressions and body language. They tell us important things about our relationships, lifestyles, desires and identities. Feelings of this sort present themselves to our conscious selves, such that we actually notice them, even if we can’t control them. Emotions can now be captured and algorithmically analysed (“sentiment analysis”) thanks to the behavioural data that digital technologies collect. And yet feelings of this sort are not welcome everywhere. In many contexts, an accusation of being “emotional” carries the implication that someone has lost objectivity and given way to irrational forces.

Feelings are how we orient ourselves, while also providing a reminder of shared humanity. Our capacity to feel pain, empathy and love is fundamental to how and why we care about each other. But as examples such as the Oxford Circus panic demonstrate, survival instincts and nerves are not always reliable. The information feelings convey in the moment can conflict starkly with the facts that are subsequently established. The crucial quality of feelings – their immediacy – is also what makes them potentially misleading, spawning overreactions and fear. Unscrupulous politicians and businesses have long exploited our instincts and emotions to convince us to believe or buy things that, on more careful reflection, we needn’t have done. Real-time media, available via mobile technologies, exacerbate this potential, meaning that we spend more of our time immersed in a stream of images and sensations, with less time for reflection or dispassionate analysis. If politics and public debate have become more emotional, as so many observers have claimed, this is as much a reflection on the speed and relentlessness of current media technologies as anything else.

During the 17th century, a number of European scholars produced ideas and institutions that aimed to regulate feelings, on the basis that they were untrustworthy and possibly dangerous. The French philosopher René Descartes treated physical sensations with great suspicion, in contrast to rational principles belonging to the mind. The English political theorist Thomas Hobbes argued that the central purpose of the state was to eradicate feelings of mutual fear and suspicion that could otherwise trigger violence. In the same era, pioneering communities of merchants and scientists introduced strict new rules for how their impressions should be recorded and spoken of, to avoid exaggeration and distortion, using numbers and public record-keeping. They would later become known as experts, and their ability to keep personal feelings separate from their observations was one of their distinguishing traits.

Some feelings have greater political potency than others. Feelings of nostalgia, resentment, anger and fear have disrupted the status quo

This era produced the intellectual building blocks of the modern age. Contemporary notions of truth, scientific expertise, public administration, experimental evidence and progress are all legacies of the 17th century. The elevation of reason above feeling was hugely productive, indeed world-changing in its implications. And yet it wasn’t simply knowledge that was being sought; it was also peace. To this day, much of the value of objectivity in public life, as manifest in statistics or news reports, is that it provides a basis for agreement among people who otherwise have little in common. A society that recognises the authority of facts must also establish certain professions and institutions that are beyond the fray of politics, sentiment or opinion.

That 17th-century project has run aground, with the results we see around us today. Those very same professions on which the modern technocratic edifice was first built are now lumped together as a “liberal elite”. Experts and facts no longer seem capable of settling arguments to the extent that they once did, while trust in the media in particular is in free fall. Objective claims about the economy, society, the human body and even nature can no longer be so successfully insulated from emotions and identities, whether on the left or the right. Consequently, the governmental institutions of the EU and Washington, DC are widely viewed as centres of elite privilege, which serve themselves rather than the public.

Appealing to sentiment … the 2017 March for Science in Washington, DC. Photograph: Paul Morigi/Getty Images

Events such as the “March for Science” which took place in March 2017 to defend science against political attacks showcase experts engaged in their own form of political mobilisation, appealing to public sentiment like any other movement. When technocrats do attempt to remain dispassionate, they run the risk of looking smug or aloof. This indicates that the politics of feeling is now unavoidable, whether one likes that or not.

Some feelings have greater political potency than others. Feelings of nostalgia, resentment, anger and fear have disrupted the status quo. Populist uprisings, as manifest in the victories of Donald Trump, the Brexit campaign and nationalist surges across Europe, are cases of this, and have been widely criticised for their denigration of expertise and harnessing of emotional discontents. But these are symptoms of a problem and not a cause. Individual leaders and campaigns will come and go, but the conditions that enabled them will endure. Rather than simply hurl more facts at these disturbances, we would do better to understand their underlying drivers. It’s no good simply declaring that experts are in possession of truth, and must therefore be obeyed. We need to understand why the ideal of expertise has lost credibility.

The aspiration to “objectivity” is in particular trouble in the economic realm. But there are good reasons why this is so. Objective indicators of progress, such as GDP growth, conceal deep fractures within society. In the US, for example, half of the population has experienced no real income growth since the 1970s, making the very notion of economic “growth” a lie. Unemployment is falling across much of Europe and the US at the moment, but this much-celebrated fact conceals the rising number of people who are being eliminated (or absenting themselves) from the labour market for reasons of ill-health, exhaustion and addiction, or taking jobs that are far inferior to ones they held in the past.

The resentment that has been directed against “liberal elites” living in major cities is not simply irrational but reflects some basic realities about how people experience inequality as a moral force, indeed a judgment. The economy isn’t just a domain for the satisfaction of objective needs and demands (as economists see it) but also one where we acquire status and self-respect. This is true the whole way across the income spectrum. Feelings of resentment or anger are not the preserve of some mythical white working class.

Yet certain demographics clearly experience far greater cultural and emotional distance from “liberal elites” than others. As party politics becomes more professionalised, it becomes harder to see any marked difference between elected representatives and their expert advisers; the state looks to many like a game being played by insiders.

Rising populism … Ukip leader Nigel Farage in Birmingham, 2016, campaigns to leave the EU.

Photograph: Christopher Furlong/Getty Images

Feelings of disenfranchisement are not merely economic, but have acquired a bodily and existential dimension: people’s lives are being shaped by divergent health, life expectancy and encounters with physical and psychological pain. Pessimism emanates most strongly from bodies that are ageing faster and suffering more – further evidence that we cannot tidily separate how things appear to our minds from how they feel to our bodies.

One could leave the story there, and simply lament the decline of modern reason, as if emotions have overwhelmed the citadel of truth like barbarians. The most vehement defenders of scientific rationality claim that alien forces – liars, demagogues, Kremlin trolls, “postmodernists” or the uneducated public – have been granted too much power, and need to be eliminated from politics all over again.

In addition to its naivety regarding the roots of populism, this response ignores a second pivotal historical development that is no less important for shaping the modern world. The desire to harness emotions and physical instincts for political purposes also has a long history, producing its own centres of elite control, but with one crucial difference: it operates in the service of conflict rather than of peace. At the height of the Enlightenment, as reason appeared to be triumphing once and for all, the French Revolution demonstrated the immense military power that could be unleashed by popular sentiment.

Modern warfare creates miasmas of emotion, information, misinformation, deception and secrecy. It mobilises infrastructure, civilian populations, industry and intelligence services in innovative ways. The rise of aerial warfare meant that problems of civilian morale and real-time decision making acquired greater urgency, producing new techniques for gauging and managing popular sentiment and sensing incoming threats. It was this paranoia that led to the invention of the digital computer and later the internet. During wartime, information becomes valued for its speed and impact as much as its public credibility. One of the principle justifications that the George W Bush administration gave for water-boarding was that information needed to be extracted from prisoners quickly, before it was too late. This is a whole new way of handling the question of truth, which often runs entirely counter to the original scientific ideal of reason and expertise.

Recently this spirit of warfare has been fed into civilian life as the emphasis on “real time” knowledge has become a feature of the business world, of the financial sector and the big Silicon Valley companies. The speed of knowledge and decision making becomes crucial, and public agreement is sidelined in the process. Rather than trusting experts, on the basis that they are neutral and outside the fray, we have come to rely on services that are fast, but whose public status is unclear.

The ideology of entrepreneurship is a contributory factor here. Since the 1970s, organisations have been exhorted to become more “agile” in the face of change, while business “disruptors” focus on seeking to exploit new knowledge and ideas before their rivals. Speed is of the essence. Among traders, whose skill lies in exploiting minuscule price movements, sometimes just fractions of a second before others. Marketing analysts aim at detecting trends as early as possible. These are exercises in feeling and sensing, rather than of reasoning. And yet nobody ever accused an advertising agency of being “post-truth”.

The spread of post-traumatic stress disorder shows how symptoms of com­bat are becoming features of civilian life

The promise of expertise, first made in the 17th century, is to provide us with a version of reality that we can all agree on. The promise of digital computing is to maximise sensitivity to a changing environment. Timing becomes everything. Experts produce facts; Google and Twitter offer trends. As the objective view of the world recedes, it is replaced by intuition as to which way things are heading now. This nervous state offers more emotional stimulation and sensitivity, but for the same reason it is unsettling and disruptive of peaceful situations. Meanwhile the question of who might be seeking to trigger specific feelings and why lurks in the background.

The ultimate danger of this condition is the one identified by Hobbes in the 17th century. Telling people that they are secure is of limited value if they sense that they are in situations of danger. The spread of post-traumatic stress disorder (a syndrome first associated with war) is one example of how symptoms of combat are becoming features of civilian life. Symptoms arise due to threats that are constant and overpowering, which can just as easily mean school bullying or controlling relationships. Wherever one stands on debates surrounding “free speech”, “trigger warnings” or “safe spaces”, it is unarguable that the boundary between “civil” and “violent” conduct has become much harder to draw – an effect of neuroscientific progress as much as of identity politics. The sudden appearance of “weaponisation” as a concept (for instance that Facebook has been “weaponised” by political campaigns) shows that once-peaceful relations are being reconceived as violent. And as the metaphor of “war” spreads (as in the “war on drugs”, “culture war” or “information war”), so trust in the institutions that make up civil society deteriorates. For those intent on sabotage, this is entirely the point.

When reason itself is in peril, there is an understandable instinct to try to revive or rescue something from the past. It has become a cliche to celebrate the rugged individualism, cold rationality and truth-seeking courage of the scientific pioneers. But in our current age, when intelligence and calculation are performed faster and more accurately by machines than by people, an alternative ideal is needed. Perhaps the great virtue of the scientific method is not that it is smart (which is now an attribute of phones, cities and fridges) but that it is slow and careful. Maybe it is not more intelligence that we need right now, but less speed and more care, both in our thinking and our feeling. After all, emotions (including anger) can be eminently reasonable, if they are granted the time to be articulated and heard. Conversely, advanced intelligence can be entirely unreasonable, when it moves at such speed as to defy any possibility of dialogue.

Democracies are being transformed by the power of feeling in ways that cannot be ignored or reversed. Separating rationality from emotion is no longer possible. This is our reality now. We can’t reverse history, and nor can we circumvent it; this new era needs to be traversed with unusual judgment and care. Rather than denigrate the influence of feelings in politics today, we might need to get better at listening to and learning from them •

• Nervous Statesis published by Cape on 20 September. William Davies is speaking at a Guardian Live event on 14 November.