On 11 October 1726, Benjamin Franklin stepped off the Berkshire and breathed in ‘the fine weather’ of Philadelphia. After spending two years in London learning the printing trade, he had crossed back over the Atlantic on a 12-week voyage that left him nauseated but craving the comforts of America. Within three years he would be publishing The Pennsylvania Gazette, a popular daily newspaper, followed by the indispensable Poor Richard’s Almanack. But on that fine October day, the 20-year-old had another idea – an idea that had him scurrying to his room to find his quill pen and a bottle of red ink.

With these, he sketched a chart – the days of the week on top and 13 ‘virtues’ on the side – which he would use to test his personal growth. ‘I conceiv’d the bold and arduous project of arriving at moral perfection’, Franklin wrote in his Autobiography. ‘I wish’d to live without committing any fault at any time; and to conquer all that either natural inclination, custom, or company might lead me into.’ The virtues he listed were temperance, silence, order, resolution, frugality, industry, sincerity, justice, moderation, cleanliness, tranquility, chastity and humility.

Franklin quickly found that he was less than perfect. Black dots began poxing his chart. On the first Sunday of his exercise he twice betrayed his virtue of silence (with ‘a habit I was getting into: prattling, punning, and joking, which only made me acceptable to trifling company’); and once his virtue of order. The following day he violated silence and order yet again, but this time he also failed to be frugal (‘to waste nothing’). Tuesday saw the same breaches of virtue as well as a failing of resolution.

Throughout that week and the next, Franklin would betray all of his virtues, perhaps most dubiously that of chastity. This was not a new problem: while working as a printer in London, he thought his marriage prospects dim, thanks to his getting frequently caught up in ‘intrigues with low women that fell in my way’. Fell in his way, indeed.

What Franklin began to learn was that attempting total perfection was futile. He gave up his chart, thinking it better to allow himself a few faults so as to be near perfect, rather than trying so hard for unattainable heights that any slight failure derailed his entire week. ‘A speckled axe is best’, he concluded. ‘A benevolent man should allow a few faults in himself.’

It is a common belief that to achieve a goal one must work at it constantly – not taking a circuitous path towards it when a straight one is available. Thus the Overeaters Anonymous organisation, the Atkins diet, the South Beach diet, and so on, ban a variety of ‘bad foods’; financial planners would probably advise clients against going to fancy restaurants while saving up to buy a house or car; a pastor would seek to dissuade his congregation from sin, no matter how minor. In order to achieve a goal, the thinking goes, one must not deviate from the straightest course; to allow for mistakes or failures is to torpedo your chances of attaining your goal.

And yet a new school of thinking is challenging these received ways and arguing that straying from the path, even engaging in hedonistic behaviour, might be the surest way to success.

Rita Coelho do Vale is an assistant professor at the Católica Lisbon School of Business and Economics, where she researches the human decision-making process with respect to self-regulation. She says that we not only can but should engage in behaviour antithetical to our ultimate goals.

In experiments conducted with Rik Pieters and Marcel Zeelenberg, and published in January 2016 in the Journal of Consumer Psychology, do Vale surveyed the way people go about achieving their goals. She concluded that it is better to make plans to fail intermittently – to splurge on occasional luxuries when saving for a house; to have a slice of chocolate cake when trying to shed a few pounds – than to end up failing anyway and getting so demoralised you give up your goal altogether.

‘It’s something that’s so obvious, but no one has ever studied these phenomena,’ do Vale told me. ‘We all plan for breaks during the day – coffee, a nap – and we know that we will feel better after these rests. But with goals we simply don’t think like this.’

Do Vale conducted a pair of diet-related experiments. A ‘straight striving group’ was asked to adhere to a strict regimen of 1,500 calories per day with limited food choices, while an ‘intermittent striving’ group was given an even stricter diet of 1,300 calories with limited choices; however, after six days of strict dieting the second group was allowed one day of 2,700 calories with unlimited food choices. Do Vale found that the ‘intermittent’ strivers had higher self-regulatory abilities, while generating a greater variety of strategies to overcome food temptations: they were more motivated to see the diet through. Participants in the ‘straight striving’ group, meanwhile, were more likely to quit the diet and report emotional setbacks when they accidentally overate. It follows that, so long as it is planned, it is often good to be bad. ‘The only way to get rid of temptation,’ Oscar Wilde wrote in The Picture of Dorian Gray, ‘is to yield to it.’

In June 2007, Angela Duckworth published a revolutionary study, where she found that the personal quality of ‘grit’ was the single most important factor in success – more important even than socioeconomic background. The world of pop psychology was set ablaze.

Duckworth, a professor at the University of Pennsylvania, with a MacArthur Fellowship under her belt (largely due to her grit-related findings), has become an in-demand public speaker and a preacher of meritocratic ascension: she espouses the idea that success is about effort and anyone, no matter where they’re from, can get to the top. It’s a theory tailor-made for the middle and upper classes: ‘We’re here because we tried; you’re there because you didn’t.’ In Grit: the Power and Passion of Perseverance, to be published this May, Duckworth supports her findings with rousing anecdotes. At the West Point military academy, a cadet’s ‘grit’ score was a better predictor of his success in the Beast Barracks (a gruelling summer training course) than his fitness, leadership ability, or intelligence; at the Scripps National Spelling Bee, contestants with the highest grit score were the most likely to get to the final round, regardless of intelligence or initial spelling ability.

Most people aren’t extremely gritty; they won’t be able to study 15 hours a day for a spelling bee, or complete punishing military training in the summer heat

This ‘grit score’, determined via a test created by Duckworth, is based on answers to questions concerning diligence, seeing tasks through, hard work, and not being discouraged by setbacks. ‘Cheat days’ are unacceptable for the high-grit person; but if she does somehow stumble, she’s able to get back up and continue forward.

Duckworth’s findings are relentless. To a certain extent she’s right: people who are able to persevere despite repeated failure do tend eventually to find success. Yet this approach to goal-completion and this negative view of setbacks (they are to be overcome, not planned or revelled in) puts this version of success out of most people’s reach.

The truth is, most people aren’t extremely gritty; they won’t be able to study for 15 hours a day for a spelling bee, or complete punishing military training courses in the summer heat. And not even the grittiest are guaranteed success. In fact, the mindset needed to maintain persistent forward motion can be its own setback. People who are obsessive and who want the very best for themselves tend to be the grittiest; they also tend, as University of Texas psychiatrist Monica Ramirez Basco writes in her book Never Good Enough (2000), to be ‘more vulnerable to depression when stressful events occur’.

Plus, much as we may want to achieve our goals – and be willing to work for them – there are limits to our capacity for work and will. That’s because willpower is a finite resource, according to Roy Baumeister, a professor of psychology at Florida State University, who coined the term ‘ego depletion’. Ego depletion (or a dwindling reserve of willpower) is the reason that you may feel less keen to exercise after a hard day at the office; it’s the reason poorer people, after expending energy on finding the best price on basic goods at the grocery store, may then buy bags of Skittles and lowbrow magazines at the checkout counter. You only have so much willpower to use before you need to take a break from decision-making and let it replenish. In Baumeister’s principal study on willpower, people who were told to give a speech advocating beliefs contrary to their own were less able to complete a difficult puzzle afterwards, implying that psychological stress had depleted their reserve of willpower.

But why do some people have significantly more willpower – and thus the potential for higher grit – than others? It might boil down to the types and number of decisions they must make each day, factors that are primarily influenced by one’s socioeconomic class.

‘Slack’, which allows a person to use more of their cognitive and emotional resources, comes from having a cushier social and financial safety net, according to Sendhil Mullainathan, a professor of economics at Harvard University, and Eldar Shafir, a behavioural scientist at Princeton University. Slack is often a better indicator of potential success than grit. It’s the reason the impoverished single mother, gritty and hardworking though she might be, is likely to have a tougher time succeeding than a young man from an affluent family. Her relative lack of slack means she has less room for error; even if she is equally good at recovering from setbacks (the quintessence of grit), she will simply face both more arduous and more numerous setbacks, leading to a faster depletion of willpower.

You soon see how privilege can exert influence in goal-driven behaviours. In this light, the notion that hard work and passion are all that is necessary for success begins to seem woefully naïve. In almost every case, but particularly where slack is in short supply, it’s advisable to plan for a setback. ‘It’s important to plan in advance to fail,’ do Vale told me. ‘Perhaps we should call failure something different – a moment of indulgence, a moment of rest, a saving of willpower.’

Epicurus understood that the expectation of future pleasure is a pleasure in itself. Taking up his ideas, the Enlightenment philosopher Jeremy Bentham noted: ‘Nature has placed mankind under the governance of two sovereign masters, pain and pleasure.’ Yet pleasure, for Epicurus and Bentham, was defined not by sensation or excitement but by both the absence of pain and the expectation of its absence. For Freud, the ‘pleasure principle’ described the active pursuit of pleasure; but in both cases, pain and pleasure are binary feelings: while a person might still have to physically endure pain, by looking forward to a lack of pain in the future – that is, pleasure – she can be sufficiently distracted from her current bodily discomfort.

The history of exaggerated pleasure is generally a response to a perceived deprivation. That is, pleasure is relative. For the late 19th-century French Decadents, for instance, whose works might seem salacious, pornographic, unnecessary (reminiscent of the Marquis de Sade), their exaggerated pleasure came to embody a bolder meaning when juxtaposed with the bourgeois values that brought levels of financial inequality and deprivation that had been unheard of since the Revolution.

To take a more prosaic example, consuming 2,300 calories might not be pleasurable to someone who consumes that on a daily basis; however it quickly becomes pleasurable when one is accustomed to eating just 1,300 calories per day. A fancy restaurant means very little if every night is spent at august tables, and a good deal more when one has been consuming dinners comprised chiefly of ramen. Therefore to plan hedonistic setbacks on your way to achieving a goal is to convert an otherwise painful process into a more pleasurable one. ‘The simple act of knowing they would have a moment of pleasure in the future made participants more persistent towards their goals’, said do Vale.

Pleasure, however, is a particularly slippery concept. Surely the pursuit of pleasure – and avoidance of pain – does not exclusively inform our every move? Is that what all of our goals come down to – maximising pleasure, minimising pain? In De Anima, Aristotle claimed animals desire things and with this desire they are given movement – a lion desires food so he runs for a gazelle. But for human beings, Aristotle says, reason also plays into our pursuit of a goal.

Humans use reason to shape how they imagine a useful object of desire. With reason and desire working in tandem we choose and pursue our goals. In Phaedrus, Plato said the soul is guided by a dark horse of passion and a white horse of reason. Socrates agreed, but said the white horse is of greater importance – we must use reason to pursue the right things; to let desire reign over reason is to chase the eventually meaningless and temporal.

Franklin eventually gave up on his virtue chart. The single fault behind all his other faults was pride

There are un-pleasurable, even painful aspects to pursuing most goals, and so we must be clever about what we choose to go after. To go wherever desire and pleasure whisk us is to fall into the trap of chasing things we want in the immediate moment but may care nothing about in the longer term. Zooming out on our lives, it is fascinating to see that both our goals, and the ways in which we set out to achieve them, so often go unexamined. Why do we want what we want? Why do we think a relentless pursuit forward is the surest way to succeed? We so seldom interrogate our desires and the best ways to achieve them – especially when those methods cut against the grain of what we’ve grown accustomed to.

After years of striving for perfection, Franklin eventually gave up on his virtue chart. He noted that the single fault behind all his other faults was pride. ‘In reality, there is, perhaps, no one of our natural passions so hard to subdue as pride’, he wrote. ‘Disguise it, struggle with it, beat it down, stifle it, mortify it as much as one pleases, it is still alive…even if I could conceive that I had completely overcome it, I should probably be proud of my humility.’

That he ever thought he could achieve perfection, without setbacks, without respites, Franklin admitted, was his gravest error. He had been naïve. And prideful. Only decades later, while writing his autobiography, did he realise that his goals could not be attained just by trying hard, by going at them again and again, without rest or leaving a place for pleasure: ‘the mere speculative conviction that it was our interest to be completely virtuous was not sufficient to prevent our slipping.’ He saw that pursuing his truest goals would take more than pure desire. It would also take reason. It would take a plan.