The eternal human struggle to live meaningfully in the face of inevitable death entered its newest phase one Monday in the summer of 2007, when employees of Google gathered to hear a talk by a writer and self-avowed geek named Merlin Mann. Their biggest professional problem was email, the digital blight that was colonising more and more of their hours, squeezing out time for more important work, or for having a life. And Mann, a rising star of the “personal productivity” movement, seemed like he might have found the answer.

He called his system “Inbox Zero”, and the basic idea was simple enough. Most of us get into bad habits with email: we check our messages every few minutes, read them and feel vaguely stressed about them, but take little or no action, so they pile up into an even more stress-inducing heap. Instead, Mann advised his audience that day at Google’s Silicon Valley campus, every time you visit your inbox, you should systematically “process to zero”. Clarify the action each message requires – a reply, an entry on your to-do list, or just filing it away. Perform that action. Repeat until no emails remain. Then close your inbox, and get on with living.

“It was really just a way of saying, ‘I suck at email, and here’s stuff that makes me suck less at it – you may find it useful,’” Mann recalled later. But he had stumbled on a rich seam of societal anxiety. Hundreds of thousands of people watched his talk online, and Inbox Zero spawned countless blog posts, along with books and apps. It was the Atkins diet for nerds: if you weren’t doing it yourself, you almost certainly knew someone who was. Mann’s followers triumphantly posted screenshots of their empty inboxes; the New Yorker, discerning his increasingly cult-like following, described his system as “halfway between Scientology and Zen”. (The New York Post called it bullshit.)

If all this fervour seems extreme – Inbox Zero was just a set of technical instructions for handling email, after all – this was because email had become far more than a technical problem. It functioned as a kind of infinite to-do list, to which anyone on the planet could add anything at will. For the “knowledge workers” of the digital economy, it was both metaphor and delivery mechanism for the feeling that the pressure of trying to complete an ever-increasing number of tasks, in a finite quantity of time, was becoming impossible to bear.

Most of us have experienced this creeping sense of being overwhelmed: the feeling not merely that our lives are full of activity – that can be exhilarating – but that time is slipping out of our control. And today, the personal productivity movement that Mann helped launch – which promises to ease the pain with time-management advice tailored to the era of smartphones and the internet – is flourishing as never before. There are now thousands of apps in the “productivity” category of the Apple app store, including software to simulate the ambient noise of working in a coffee shop (this has been shown, in psychology experiments, to help people focus on work), and a text editor that deletes the words you have written if you don’t keep typing fast enough.

The quest for increased personal productivity – for making the best possible use of your limited time – is a dominant motif of our age. Two books on the topic by the New York Times journalist Charles Duhigg have spent more than 60 weeks on the US bestseller lists between them, and the improbable titular promise of another book, The Four Hour Work Week, has seduced a reported 1.35m readers worldwide. There are blogs offering tips on productive dating, and on the potential result of productive dating, productive parenting; signs have been spotted in American hotels wishing visitors a “productive stay”. The archetypal Silicon Valley startup, in the last few years, has been one that promises to free up time and mental capacity by eliminating some irritating “friction” of daily life – shopping or laundry, or even eating, in the case of the sludgy, beige meal replacement Soylent – almost always for the purpose of doing more work.

And yet the truth is that more often than not, techniques designed to enhance one’s personal productivity seem to exacerbate the very anxieties they were meant to allay. The better you get at managing time, the less of it you feel that you have. Even when people did successfully implement Inbox Zero, it didn’t reliably bring calm. Some interpreted it to mean that every email deserved a reply, which only shackled them more firmly to their inboxes. (“That drives me crazy,” Mann says.) Others grew jumpy at the thought of any messages cluttering an inbox that was supposed to stay pristine, and so ended up checking more frequently. My own dismaying experience with Inbox Zero was that becoming hyper-efficient at processing email meant I ended up getting more email: after all, it’s often the case that replying to a message generates a reply to that reply, and so on. (By contrast, negligent emailers often discover that forgetting to reply brings certain advantages: people find alternative solutions to the problems they were nagging you to solve, or the looming crisis they were emailing about never occurs.)

The allure of the doctrine of time management is that, one day, everything might finally be under control. Yet work in the modern economy is notable for its limitlessness. And if the stream of incoming emails is endless, Inbox Zero can never bring liberation: you’re still Sisyphus, rolling his boulder up that hill for all eternity – you’re just rolling it slightly faster.

Two years after his Google talk, Mann released a rambling and slightly manic online video in which he announced that he had signed a contract for an Inbox Zero book. But his career as a productivity guru had begun to stir an inner conflict. “I started making pretty good money from it” – from speaking and consulting – “but I also started to feel terrible,” he told me earlier this year. “This topic of productivity induces the worst kind of procrastination, because it feels like you’re doing work, but I was producing stuff that had the express purpose of saying to people, ‘Look, come and see how to do your work, rather than doing your work!’”

The book missed its publication date. Fans started asking questions. Then, after two more years, Mann published a self-lacerating essay in which he abruptly announced that he was jettisoning the project. It was the 3,000-word howl of a man who had suddenly grasped the irony of missing morning after morning with his three-year-old daughter because he was “typing bullshit that I hoped would please my book editor” about how to use time well. He was guilty, he declared, of “abandoning [my] priorities to write about priorities … I’ve unintentionally ignored my own counsel to never let your hard work fuck up the good things.” He hinted that he might write a different kind of book instead – a book about stuff that really mattered – but it never appeared. “I’m mostly out of the productivity racket these days,” Mann told me. “If you’re just using efficiency to jam more and more stuff into your day … well, how would you ever know that that’s working?”

It’s understandable that we respond to the ratcheting demands of modern life by trying to make ourselves more efficient. But what if all this efficiency just makes things worse?

Facebook Twitter Pinterest Illustration by Pete Gamlen

Given that the average lifespan consists of only about 4,000 weeks, a certain amount of anxiety about using them well is presumably inevitable: we’ve been granted the mental capacities to make infinitely ambitious plans, yet almost no time at all to put them into practice. The problem of how to manage time, accordingly, goes back at least to the first century AD, when the Roman philosopher Seneca wrote On The Shortness of Life. “This space that has been granted to us rushes by so speedily, and so swiftly that all save a very few find life at an end just when they are getting ready to live,” he said, chiding his fellow citizens for wasting their days on pointless busyness, and “baking their bodies in the sun”.

Clearly, then, the challenge of how to live our lives well is not a new one. Still, it is safe to say that the citizens of first-century Rome didn’t experience the equivalent of today’s productivity panic. (Seneca’s answer to the question of how to live had nothing to do with becoming more productive: it was to give up the pursuit of wealth or high office, and spend your days philosophising instead.) What is uniquely modern about our fate is that we feel obliged to respond to the pressure of time by making ourselves as efficient as possible – even when doing so fails to bring the promised relief from stress.

It is either impossible, or at least usually feels impossible, to cut down on work in exchange for more time

The time-pressure problem was always supposed to get better as society advanced, not worse. In 1930, John Maynard Keynes famously predicted that within a century, economic growth would mean that we would be working no more than 15 hours per week – whereupon humanity would face its greatest challenge: that of figuring out how to use all those empty hours. Economists still argue about exactly why things turned out so differently, but the simplest answer is “capitalism”. Keynes seems to have assumed that we would naturally throttle down on work once our essential needs, plus a few extra desires, were satisfied. Instead, we just keep finding new things to need. Depending on your rung of the economic ladder, it’s either impossible, or at least usually feels impossible, to cut down on work in exchange for more time.

Arguably the first time management guru – the progenitor of the notion that personal productivity might be the answer to the problem of time pressure – was Frederick Winslow Taylor, an engineer hired in 1898 by the Bethlehem Steel Works, in Pennsylvania, with a mandate to improve the firm’s efficiency. “Staring out over an industrial yard that covered several square miles of the Pennsylvania landscape, he watched as labourers loaded 92lb [iron bars] on to rail cars,” writes Matthew Stewart, in his book The Management Myth. “There were 80,000 tons’ worth of iron bars, which were to be carted off as fast as possible to meet new demand sparked by the Spanish-American war. Taylor narrowed his eyes: there was waste here, he was certain.”

The Bethlehem workers, Taylor calculated, were shifting about 12.5 tons of iron per man per day – but predictably, when he offered a group of “large, powerful Hungarians” some extra cash to work as fast as they could for an hour, he found that they performed much better. Extrapolating to a full work day, and guesstimating time for breaks, Taylor concluded, with his trademark blend of self-confidence and woolly maths, that every man ought to be shifting 50 tons per day – four times their usual amount.

Workers were naturally unhappy at this transparent attempt to pay them the same money for more work, but Taylor was not especially concerned with their happiness; their job was to implement, not understand, his new philosophy of “scientific management”. “One of the very first requirements for a man who is fit to handle pig iron,” wrote Taylor, is “that he shall be so stupid and phlegmatic that he more nearly resembles in his mental makeup the ox than any other type … he is so stupid that the word ‘percentage’ has no meaning for him.”

The idea of efficiency that Taylor sought to impose on Bethlehem Steel was borrowed from the mechanical engineers of the industrial revolution. It was a way of thinking about improving the functioning of machines, now transferred to humans. And it caught on: Taylor enjoyed a high-profile career as a lecturer on the topic, and by 1915, according to the historian Jennifer Alexander, “the word ‘efficiency’ was plastered everywhere – in headlines, advertisements, editorials, business manuals, and church bulletins.” In the first decades of the 20th century, in a Britain panicked by the rise of German power, the National Efficiency movement united politicians on left and right. (“At the present time,” the Spectator noted in 1902, “there is a universal outcry for efficiency in all the departments of society, in all aspects of life.”)

It is not hard to grasp the appeal: efficiency was the promise of doing what you already did, only better, more cheaply, and in less time. What could be wrong with that? Unless you happened to be on the sharp end of attempts to treat humans like machines – like the workers of Bethlehem Steel – there wasn’t an obvious downside.

Facebook Twitter Pinterest Illustration by Pete Gamlen

But as the century progressed, something important changed: we all became Frederick Winslow Taylors, presiding ruthlessly over our own lives. As the doctrine of efficiency grew entrenched – as the ethos of the market spread to more and more aspects of society, and life became more individualistic – we internalised it. In Taylor’s day, efficiency had been primarily a way to persuade (or bully) other people to do more work in the same amount of time; now it is a regimen that we impose on ourselves.

According to legend, Taylorism first crossed the threshold into personal productivity when Charles Schwab, the president of Bethlehem Steel, asked another consultant, a businessman named Ivy Lee, to improve his executives’ efficiency as well. Lee advised those white-collar workers to make nightly to-do lists, arranging tomorrow’s six most important tasks by priority, then to start at the top of the list next morning, working down. It’s a stretch to imagine that nobody had thought of this before. But the story goes that when Lee told Schwab to test it for three months, then pay him what he thought it was worth, the steel magnate wrote him a cheque worth more than $400,000 in today’s money – and the time management industry was up and running.

In an era of insecure employment, we must constantly demonstrate our usefulness through frenetic doing

Other gurus were to follow, writing bestsellers that modified Lee’s basic technique to incorporate the setting of long-term goals (the 1973 book How to Get Control of Your Time and Your Life, by Alan Lakein, who boasted of having advised both IBM and Gloria Steinem, and who inspired a young Bill Clinton) and spiritual values (The Seven Habits of Highly Effective People, published in 1989 by the Mormon efficiency expert Stephen Covey).

Time management promised a sense of control in a world in which individuals – decreasingly supported by the social bonds of religion or community – seemed to lack it. In an era of insecure employment, we must constantly demonstrate our usefulness through frenetic doing, and time management can give you a valuable edge. Indeed, if you are among the growing ranks of the self-employed, as a freelancer or a worker in the so-called gig economy, increased personal efficiency may be essential to your survival. The only person who suffers financially if you indulge in “loafing” – a workplace vice that Taylor saw as theft – is you.

Above all, time management promises that a meaningful life might still be possible in this profit-driven environment, as Melissa Gregg explains in Counterproductive, a forthcoming history of the field. With the right techniques, the prophets of time management all implied, you could fashion a fulfilling life while simultaneously attending to the ever-increasing demands of your employer. This promise “comes back and back, in force, whenever there’s an economic downturn”, Gregg told me.

Illustration by Pete Gamlen

Especially at the higher-paid end of the employment spectrum, time management whispers of the possibility of something even more desirable: true peace of mind. “It is possible for a person to have an overwhelming number of things to do and still function productively with a clear head and a positive sense of relaxed control,” the contemporary king of the productivity gurus, David Allen, declared in his 2001 bestseller, Getting Things Done. “You can experience what the martial artists call a ‘mind like water’, and top athletes refer to as ‘the zone’.”

As Gregg points out, it is significant that “personal productivity” puts the burden of reconciling these demands squarely on our shoulders as individuals. Time management gurus rarely stop to ask whether the task of merely staying afloat in the modern economy – holding down a job, paying the mortgage, being a good-enough parent – really ought to require rendering ourselves inhumanly efficient in the first place.

Besides, on closer inspection, even the lesser promises of time management were not all they appeared to be. An awkward truth about Taylor’s celebrated efficiency drives is that they were not very successful: Bethlehem Steel fired him in 1901, having paid him vast sums without any clearly detectable impact on its own profits. (One persistent consequence of his schemes was that they seemed promising at first, but left workers too exhausted to function consistently over the long term.)

Likewise, it remains the frequent experience of those who try to follow the advice of personal productivity gurus – I’m speaking from years of experience here – that a “mind like water” is far from the guaranteed result. As with Inbox Zero, so with work in general: the more efficient you get at ploughing through your tasks, the faster new tasks seem to arrive. (“Work expands to fill the time available for its completion,” as the British historian C Northcote Parkinson realised way back in 1955, when he coined what would come to be known as Parkinson’s law.)

Then there’s the matter of self-consciousness: virtually every time management expert’s first piece of advice is to keep a detailed log of your time use, but doing so just heightens your awareness of the minutes ticking by, then lost for ever. As for focusing on your long-term goals: the more you do that, the more of your daily life you spend feeling vaguely despondent that you have not yet achieved them. Should you manage to achieve one, the satisfaction is strikingly brief – then it’s time to set a new long-term goal. The supposed cure just makes the problem worse.

There is a historical parallel for all this: it’s exactly what happened when the spread of “labour-saving” devices transformed the lives of housewives and domestic servants across Europe and north America from the end of the 19th century. Technology now meant that washing clothes no longer entailed a day bent over a mangle; a vacuum-cleaner could render a carpet spotless in minutes.

The 2016 Time Matters conference was sparsely attended because it was August, and lots of people were on holiday

Yet as the historian Ruth Cowan demonstrates in her 1983 book More Work for Mother, the result, for much of the 20th century, was not an increase in leisure time among those charged with doing the housework. Instead, as the efficiency of housework increased, so did the standards of cleanliness and domestic order that society came to expect. Now that the living-room carpet could be kept perfectly clean, it had to be; now that clothes never needed to be grubby, grubbiness was all the more taboo. These days, you can answer work emails in bed at midnight. So should that message you got at 5.30pm really wait till morning for a reply?

One boiling weekend last summer, the impassioned members of a campaign group named Take Back Your Time gathered in a university lecture hall in Seattle, to further their longstanding mission of “eliminating the epidemic of overwork” – and, in so doing, to explore what it might mean to live a life that is not so focused on personal productivity. The 2016 Time Matters conference was a sparsely attended affair, in part because, as the organisers conceded, it was August, and lots of people were on holiday, and America’s most enthusiastically pro-relaxation organisation was hardly going to complain about that. But it was also because, these days, being even modestly anti-productivity – especially in the US – counts as a subversive stance. It is not the kind of platform that lends itself to glitzy mega-events with generous corporate sponsorship and effective marketing campaigns.

The conference-goers discussed schemes for a four-day working week, for abolishing daylight savings time, for holding elections at the weekend, and generally for making America more like countries such as Italy and Denmark. (To be a critic of America’s work culture is to constantly gaze longingly across the Atlantic, at semi-mythical versions of Scandinavia and southern Europe.) But the members of Take Back Your Time were calling for something more radical than merely more time off. They sought to question our whole instrumental attitude towards time – the very idea that “getting more done” ought to be our focus in the first place. “You keep hearing people arguing that more time off might be good for the economy,” said John de Graaf, the not-even-slightly-relaxed 70-year-old filmmaker who is the organisation’s driving force. “But why should we have to justify life in terms of the economy? It makes no sense!”

One of the sneakier pitfalls of an efficiency-based attitude to time is that we start to feel pressured to use our leisure time “productively”, too – an attitude which implies that enjoying leisure for its own sake, which you might have assumed was the whole point of leisure, is somehow not quite enough. And so we find ourselves, for example, travelling to unfamiliar places not for the sheer experience of travel, but in order to add to our mental storehouse of experiences, or to our Instagram feeds. We go walking or running to improve our health, not for the pleasure of movement; we approach the tasks of parenthood with a fixation on the successful future adults we hope to create.

In his 1962 book The Decline of Pleasure, the critic Walter Kerr noticed this shift in our experience of time: “We are all of us compelled to read for profit, party for contracts, lunch for contacts … and stay home for the weekend to rebuild the house.” Even rest and recreation, in a culture preoccupied with efficiency, can only be understood as valuable insofar as they are useful for some other purpose – usually, recuperation, so as to enable more work. (Several conference guests mentioned Arianna Huffington’s current crusade to encourage people to get more sleep; for her, it seems, the main point of rest is to excel at the office.)

If all this increased efficiency brings none of the benefits it was supposed to bring, what should we be doing instead? At Take Back Your Time, the consensus was that personal lifestyle changes would never suffice: reform would have to start with policies on vacation, maternity leave and overtime. But in the meantime, we might try to get more comfortable with not being as efficient as possible – with declining certain opportunities, disappointing certain people, and letting certain tasks go undone. Plenty of unpleasant chores are essential to survival. But others are not – we have just been conditioned to assume that they are. It isn’t compulsory to earn more money, achieve more goals, realise our potential on every dimension, or fit more in. In a quiet moment in Seattle, Robert Levine, a social psychologist from California, quoted the environmentalist Edward Abbey: “Growth for the sake of growth is the ideology of the cancer cell.”

Facebook Twitter Pinterest Illustration by Pete Gamlen

Yet if the ethos of efficiency and productivity risks prioritising the health of the economy over the happiness of humans, it is also true that the sense of pressure it fosters is not much good for business, either. This, it turns out, is a lesson business is not especially keen to learn.

“After years of consulting with Microsoft, I was suddenly persona non grata,” Tom DeMarco told me, with a note of amusement in his voice. DeMarco is a minor legend in the world of software engineering. He began his career at Bell Telephone Labs, birthplace of the laser and transistor, and later became an expert in managing complex software projects, a field notorious for spiralling costs, missed deadlines, and clashing egos. But then, in the 1980s, he committed heresy: he started arguing that ramping up the time pressure on your employees was a terrible way to drive such projects forward. What was needed, he had come to realise, was not an increased focus on using time efficiently. It was the opposite: more slack.

Thinking about time encourages clockwatching, which has been repeatedly shown to undermine the quality of work

“The best companies I visited, all through the years, were never very hurried,” DeMarco said. “Maybe they used pressure from time to time, as a sort of amusing side-effect. But it was never a constant. Because you don’t get creativity for free. You need people to be able to sit back, put their feet up, and think.” Manual work can be speeded up, at least to a certain extent, by increasing the time pressure on workers. But good ideas do not emerge more rapidly when people feel under the gun – if anything, the good ideas dry up.

Part of the problem is simply that thinking about time encourages clockwatching, which has been repeatedly shown in studies to undermine the quality of work. In one representative experiment from 2008, US researchers asked people to complete the Iowa gambling task, a venerable decision-making test that involves selecting playing cards in order to win a modest amount of cash. All participants were given the same time in which to complete the task – but some were told that time would probably be sufficient, while others were warned it would be tight. Contrary to an intuition cherished especially among journalists – that the pressure of deadlines is what forces them to produce high-quality work – the second group performed far less well. The mere awareness of their limited time triggered anxious emotions that got in the way of performance.

But worse perils await. DeMarco points out that any increase in efficiency, in an organisation or an individual life, necessitates a trade-off: you get rid of unused expanses of time, but you also get rid of the benefits of that extra time. A visit to your family doctor provides an obvious example. The more efficiently they manage their time, the fuller their schedule will be – and the more likely it is that you will be kept sitting in the waiting room when an earlier appointment overruns. (That’s all a queue is, after all: the cost of someone else’s efficiency, being shouldered by you.) In the accident and emergency department, by contrast, remaining “inefficient” in this sense is a matter of life and death. If there is an exclusive focus on using the staff’s time as efficiently as possible, the result will be a department too busy to accommodate unpredictable arrivals, which are the whole reason it exists.

A similar problem afflicts any corporate cost-cutting exercise that focuses on maximising employees’ efficiency: the more of their hours that are put to productive use, the less available they will be to respond, on the spur of the moment, to critical new demands. For that kind of responsiveness, idle time must be built into the system.

“An organisation that can accelerate but not change direction is like a car that can speed up but not steer,” DeMarco writes. “In the short run, it makes lots of progress in whatever direction it happened to be going. In the long run, it’s just another road wreck.” He often uses the analogy of those sliding number puzzles, in which you move eight tiles around a nine-tile grid, until all the digits are in order. To use the available space more efficiently, you could always add a ninth tile to the empty square. You just wouldn’t be able to solve the puzzle any more. If that jammed and unsolvable puzzle feels like an appropriate metaphor for your life, it’s hard to see how improving your personal efficiency – trying to force yet more tiles on to the grid – is going to be much help.

At the very bottom of our anxious urge to manage time better – the urge driving Frederick Winslow Taylor, Merlin Mann, me and perhaps you – it’s not hard to discern a familiar motive: the fear of death. As the philosopher Thomas Nagel has put it, on any meaningful timescale other than human life itself – that of the planet, say, or the cosmos – “we will all be dead any minute”. No wonder we are so drawn to the problem of how to make better use of our days: if we could solve it, we could avoid the feeling, in Seneca’s words, of finding life at an end just when we were getting ready to live. To die with the sense of nothing left undone: it’s nothing less than the promise of immortality by other means.

But the modern zeal for personal productivity, rooted in Taylor’s philosophy of efficiency, takes things several significant steps further. If only we could find the right techniques and apply enough self-discipline, it suggests, we could know that we were fitting everything important in, and could feel happy at last. It is up to us – indeed, it is our obligation – to maximise our productivity. This is a convenient ideology from the point of view of those who stand to profit from our working harder, and our increased capacity for consumer spending. But it also functions as a form of psychological avoidance. The more you can convince yourself that you need never make difficult choices – because there will be enough time for everything – the less you will feel obliged to ask yourself whether the life you are choosing is the right one.

Personal productivity presents itself as an antidote to busyness when it might better be understood as yet another form of busyness. And as such, it serves the same psychological role that busyness has always served: to keep us sufficiently distracted that we don’t have to ask ourselves potentially terrifying questions about how we are spending our days. “How we labour at our daily work more ardently and thoughtlessly than is necessary to sustain our life because it is even more necessary not to have leisure to stop and think,” wrote Friedrich Nietzsche, in what reads like a foreshadowing of our present circumstances. “Haste is universal because everyone is in flight from himself.”

You can seek to impose order on your inbox all you like – but eventually you’ll need to confront the fact that the deluge of messages, and the urge you feel to get them all dealt with, aren’t really about technology. They’re manifestations of larger, more personal dilemmas. Which paths will you pursue, and which will you abandon? Which relationships will you prioritise, during your shockingly limited lifespan, and who will you resign yourself to disappointing? What matters?

For Merlin Mann, consciously confronting these questions was a matter of realising that people would always be making more claims on his time – worthy claims, too, for the most part – than it would be possible for him to meet. And that even the best, most efficient system for managing the emails they sent him was never going to provide a solution to that. “Eventually, I realised something,” he told me. “Email is not a technical problem. It’s a people problem. And you can’t fix people.”

• Main illustration: Pete Gamlen

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.

This article contains affiliate links, which means we may earn a small commission if a reader clicks through and makes a purchase. All our journalism is independent and is in no way influenced by any advertiser or commercial initiative. By clicking on an affiliate link, you accept that third-party cookies will be set. More information.