Arriving at the chess board is like entering an eagerly anticipated party. All my old friends are there: the royal couple, their associates, the reassuringly straight lines of noble infantry. I adjust them, ensuring that they are optimally located in the centre of their starting squares, an anxious fidgeting and tactile caress. I know these pieces, and care about them. They are my responsibility. And I’m grateful to my opponent for obliging me to treat them well on pain of death.

In many ways, I owe chess everything. Since the age of five, the game has been a source of friendship, refuge and growth, and I have been a grandmaster for 20 years. The lifelong title is the highest awarded to chess players, and it is based on achieving three qualifying norms in international events that are often peak performances, combined with an international rating reflecting a consistently high level of play – all validated by FIDE, the world chess federation. There are about 1,500 grandmasters in the world. At my peak, I was just outside the world top 100, and I feel some gentle regret at not climbing even higher, but I knew there were limits. Even in the absence of a plan A for my life, chess always felt like plan B, mostly because I couldn’t imagine surrendering myself to competitive ambition. I have not trained or played with serious professional intent for more than a decade, and while my mind remains charmed by the game, my soul feels free of it.

In recent years, I have worked in academic and public policy contexts, attempting to integrate our understanding of complex societal challenges with our inner lives, while also looking after my two sons. I miss many things about not being an active player. I miss the feeling of strength, power and dignity that comes with making good decisions under pressure. I miss the clarity of purpose experienced at each moment of each game, the lucky escapes from defeat, and the thrill of the chase towards victory. But, most of all, I miss the experience of concentration.

I can still concentrate, of course, but not with the same reliability and intensity that a life of professional chess affords. In fact, from a distance, chess looks to me suspiciously like a socially permissible pretext to concentrate for several hours at a time. In The Island from the Day Before (1994), Umberto Eco composes a love letter that includes the line: ‘[O]nly in your prison does [my heart] enjoy the most sublime of freedoms’ – that could be said of chess, too, and the experience of concentration is what makes it possible. I believe concentration is a defining feature of a fulfilling life, a necessary habit of mind for a viable civilisation, and that chess can teach us more about what concentration really means.

Any skilled endeavour entails concentration, but chess is unusual in requiring that we concentrate not for a few minutes at a time, but for several hours at a time, within tournaments, for days at a time, and within careers, for years at a time. Concentration is the sine qua non of the chess experience.

In chess, concentration usually unfolds in quick succession through perceiving, desiring and searching. But it’s recursive, so I often find something I didn’t expect in a way that leads me to see my position differently and want something else from it. My perception is pre-patterned through years of experience, so I don’t see one square or piece at a time. Instead, I see the whole position as a situation featuring relationships between pieces in familiar strategic contexts; a castled king, a fianchettoed bishop, a misplaced knight, an isolated pawn; it’s a kind of conceptual grammar. The meaning of the position is embedded in those patterns, partly revealed and partly concealed, and my search to do the right thing feels fundamentally aesthetic in nature.

I could describe the feeling as a kind of evaluative hunting – not so much for a particular target, but for trails of ideas that look right and feel right. I am drawn towards some transfigurations of the patterns that make me look deeper, and repelled by others. Good moves have the qualities of truth and beauty. They are discoveries of how things are, and should be.

However, chess invites me to deepen my concentration a few centimetres away from another being who is also trying to concentrate; someone I can smell, sense moving, and hear breathing. I often know, even like, these people, but they loom within my psyche in a relatively impersonal sense – a familiar energy, not friends as such. I sometimes think of chess opponents as psychopathic flatmates with whom I have to share a living space. They look harmless, but I know we signed the same contract that says they need to try to get inside my room, steal my possessions and hunt me down, before killing me; naturally, I am obliged to do the same to them. Together we create a story, and narrative themes such as attack and defence are both reduced and reified into particular moves with particular pieces on particular squares, which we record like stenographers, into our own arcana of algebraic notation. The climax of a game’s story might be ‘Brutal counter-attack!’ but the record merely reflects the logical power of a short sequence of moves, for instance: ‘…34. Bf3 Nh3+ 35.Kh1 Qg4!! Resigns.’

I executed the additional detail only I had seen, and he immediately resigned. I felt strong

The forces on the board are always embroiled, but concentration is particularly important when the pieces stop eyeing each other from a strategic distance, and come into direct tactical contact. At such moments, spotting a hidden detail could guarantee victory, while missing it could lead to inexorable defeat. Such details were usually a few moves away from any given position in front of me, so I would have to seek them out; while much of chess thinking has a narrative quality, this seeking called for the merciless logic of calculation. It’s about keeping track of the balance of material forces as they attempt to eliminate each other in a battle for supremacy. The process is strenuous, even painful, but learning to appreciate the beauty of uncovering the truth was critical in my advancing up the grandmaster ranks.

One of the highlights of my chess career was beating the Russian-born grandmaster Alex Yermolinsky at the World Open in Philadelphia in 2002, because it was a palpable experience of self-overcoming. ‘Yermo’ is a two-time US Champion. On paper, he was the favourite, but I’d recently been training by solving chess exercises, setting up carefully vetted positions and deciding what I would play, then comparing my thoughts with the book answer. Yermolinsky offered a pawn as bait, and I very nearly didn’t take it because doing so would allow him to play a series of forcing moves, including an elegant counterattack that appeared decisive. Looking deeper, I discovered a surprising detail right at the end of the line, in which my knight could retreat back to its original square, solving all my defensive problems and leaving me with a decisive advantage. I checked the variation just once – the wise side of neurotic! – and we briskly played straight down the line. The clock clicked gently with each move. Yermo played the impressive-looking tactic that we’d both anticipated as if it was decisive. Then I executed the additional detail only I had seen, and he immediately resigned. I felt strong.

Concentration is not always so rewarding. It comes and goes, forms and collapses, builds and then crumbles, because there is an upper limit to what players can hold in their heads at any one time. I find that I move towards my upper limit and away from it repeatedly. Peering into the unfolding position, it is as if I am driving more or less automatically, until new possibilities flash before me like bikes emerging from side-streets, and bring me back to the challenge of steering consciousness. At such moments, the edifice of thought I have built is likely to collapse. If I’m not careful, I can spend far too many minutes in this state of perpetual irresolution, seeking but not finding an answer to what is happening, because there is just too much meaning in the position for my mind to process. This challenge of learning how to hold complexity in mind and still make good decisions is pertinent not just to chess but to life more generally.

As a chess grandmaster, I find the familiar injunction to ‘Concentrate!’ a little naive. Concentration is not like a bulb that we can turn on and off with a switch, because we are not just the bulb; we are also the switcher and the switch. Humans are more like thermostats receiving and sending out signals, seeking the optimal ‘mental temperature’ as ambient conditions around and within us change, and we’re often abruptly adjusted against our will. We succeed in concentrating when we manage to convene the dispositions that matter for a task at hand – for instance, our awareness, attention, discernment and willpower – and that is possible only if the right emotions co-arise and come along for the ride.

Concentration is therefore best understood as a kind of coalescence. The ultimate aim might be single-pointed attention, but the process of concentrating is more like a method of corralling and coordinating fissiparous parts of our psyche. The best illustration of this idea is traditional forms of yoga, which is practised through seemingly elaborate asanas (postures) such as standing on your head, and forms of pranayama (breathing) in which you pump stale air out of your lungs through your nostrils as a kind of spiritual snorting. Such practices have value on their own terms, but their ultimate purpose is the experience of realisation in which our nervous systems are sufficiently settled through training that we can sit still, free of mental agitation, for more than a few seconds at a time.

When viewed this way, it is important to distinguish concentration from similar or related phenomena that provide what the US theorist Bonnitta Roy calls different ‘meta-cognitive views’ – contexts of meaning and activity that are valuable because they allow the mind to become aware of itself. Chess thinking provides a rich metacognitive context that leads me to believe that we should tease apart three notions that are related but often conflated – attention, flow and concentration. Attention is fundamentally grounded in perception (how we attend), flow is fundamentally grounded in experience (how we feel), and concentration is grounded in praxis (how we purposively coalesce).

We ask too much of attention and not enough of concentration. The recent cultural emphasis on attention risks subsuming too many variables of human experience, as if they could ever be held constant. We have to pay attention with the body, the will, the place, the mood, the memory, the moment, the relationships, the affordances, not the least the smartphone. All these variables are implicated in our capacity to attend, but they have their own kinds of agency, too, and they play with each other in unpredictable ways. The emergent properties arising from the psyche at play with itself in the world include amusement, enchantment, dissonance and distraction: these are not mere hindrances but more like a kind of data to be understood and integrated before we can exercise agency that is truly our own. We need to coalesce in order to concentrate, and concentrate to coalesce.

As the will to deepen my attention for chess was dying, a part of myself was dying, too

If we can’t concentrate, we will not be able to enjoy the state of consciousness – called flow – that is part of the chess experience. Flow was conceived and popularised by the Hungarian-American psychologist Mihaly Csikszentmihalyi, and it is a mental state characterised by intense absorption, loss of self-consciousness, goal-related feedback from the world and an altered sense of time. Flow experiences are deeply rewarding, and they arise when our skill level and challenge level are optimally matched; too little challenge and we get bored, too much and we feel anxious. Chess is a great way to access flow, yet – as a lodestar for living – flow has limitations. Mostly, it describes a quality of consciousness, not a method for obtaining it. Ultimately, flow is not a virtue but a form of pleasure. While flow is a desirable state of mind, promoting it might not lead to desirable qualities of character; just as likely it could yield an atomised society of sophisticated hedonists with gaming addictions and virtual-reality sickness.

Unlike attention or flow, concentration prompts an awareness of mood, even a commitment to meaning, and an appreciation for method. As a young player, I leant on various methods to deepen concentration, including taking long walks before games and listening to favourite music tracks. They worked mostly because the purpose of the concentration was never in doubt. However, in early December 2008, in a generic hotel room in Palma, Mallorca, I vividly remember trying to prepare for a game while feeling unusually displaced from myself. As generations of American football coaches have put it: ‘The will to win is not as important as the will to prepare to win.’ And I noticed that I had lost that. It felt as if the underlying motivational vector for concentration had collapsed; and, as the will to deepen my attention for chess was dying, a part of myself was dying, too. The will to sustain the identity that perpetuated my desire to win had gone, and I knew that it was time to concentrate on what the game symbolised rather than on the game itself.

Chess is an arena where system meets psyche, and the world needs a way to make better sense of the nature of that encounter today. To make good chess moves, you need to see the whole position in all its reverberating dynamism. But you also need to see your own mind, and know its power and limitations; the personal is indeed political, and vice-versa. In the early 21st century, the position we face includes a cascading ecological crisis (when material-intensive economic growth remains the world’s prevailing priority); the challenge of preventing mass unemployment in an age of rising AI; protecting the truth when lies are easier; more exciting and faster travel; and strengthening collaborative governance in a time of vested interests and spiritual and political alienation.

In his Utopian novel Island (1962), Aldous Huxley depicts ‘reminder birds’ called Mynahs who fly around periodically saying: ‘Attention!’ and ‘Here and now!’ to help bring the inhabitants back to themselves and the present moment. However, if Mynahs were to be released into London, New York, Delhi or Beijing today, it’s not clear what we would be asked to pay attention to or for. Today’s Mynahs are smartphone notifications, which seduce us through our weakness for novelty and coerce us through our fear of missing out, as ubiquitous advertisers, in league with psychographic profilers, harvest our attention as a commodity. Our problem today is not that we don’t or can’t pay attention, but that the systems and structures of society oblige us to pay attention so frequently and fleetingly that we cannot in fact concentrate. Lacking an ability to concentrate, it’s a struggle to construct and maintain a coherent and autonomous sense of self, which leaves us at the mercy of digital, commercial and political puppeteers. Without concentration, we are not free.

I am glad that attention is growing in importance as a political concept to enrich our understanding of freedom, and to describe the interface between self and world. However, as a chess grandmaster, I feel that the issue has been misconceived. Our challenge today is not that we all have to pay attention, but that we need to know attention from the inside, which means that we have to learn to concentrate.

There is no hope if we see each problem as a discrete issue to be analysed by a distinct discipline

Most complex problems cannot be properly understood or experienced unless we can consider several ideas and ways of thinking together at one time. However, if all we can do is simply hold ideas, we won’t be able to actually think with them or about them; we will be those thoughts but we won’t really have them. Developing concentration therefore entails developing the capacity to hold the emotional tension of mental complexity; we have to train ourselves to resist the temptation to give up, to oversimplify or project onto our perceived opponents. In his classic text, Thought as a System (1992), the US physicist and philosopher David Bohm put the challenge like this:

[T]he general tacit assumption in thought is that it’s just telling you the way things are and that it is not doing anything – that ‘you’ are inside there, deciding what to do with the information. But I want to say that you don’t decide what to do with the information. The information takes over. It runs you. Thought runs you. Thought, however, gives the false information that you are running it, that you are the one who controls thought, whereas actually thought is the one which controls each one of us.

What Bohm is alluding to is our need to find a vantage point outside whatever system of facts, associations and language forms are shaping our idea of what is happening – that kind of luminous vantage point is the achievement of concentration properly understood, and there is no quick way to get there. The essence of our thinking challenge today is that the world has different kinds of problems that are nonetheless profoundly interconnected: emotional and ecological; psychological and political; spiritual and systemic. Yet our ways of knowing and acting remain partial and fragmented.

Unless we can learn to concentrate better, we have no chance of perceiving, thinking, talking and deciding in the ways required of us in the 21st century. There is no hope for us if we start from a vantage point that sees each problem as a discrete issue, ciphered off into an expert silo to be analysed by a distinct discipline. Albert Einstein was right when he said that we cannot solve our problems with the same thinking that caused them – but, in the world as we find it, genuinely new thinking calls for a reappraisal of concentration.