There are countless stories about how we ought to live our lives. We are expected to be ambitious; to want to be wealthy, successful and well educated; to get married, be monogamous and have kids. These social narratives can make our lives easier, by providing guidelines for behaviour, and they might sometimes make us happier, too. But they are, at their heart, stories – and ones that may not have originated with present-day people in mind. As such, many of these stories end up creating a kind of social dissonance whereby, perversely, they cause more harm than good.

Since we’re talking about stories, let’s start with an experience of mine. It’s about a working-class kid who becomes a university professor and who is expected to change his behaviour in accordance with a (harmful) narrative about how academics ought to behave. A couple of years ago, I took part in an interesting panel discussion on “emotion versus reason” at the HowTheLightGetsIn festival in Hay-on-Wye. Walking across the field to get some food, I was approached by a man in his 50s. Our interaction started with him saying how much he liked my first book, Happiness By Design. Then he asked, pointedly: “But why do you have to play the working-class hero? You do it in your book and, look at you, you’re doing it now.”

Ordinarily, I would embrace being a hero of any kind but, in this context, I was made to feel like an arsehole. The man proceeded to lecture me on how “when you reach a certain level you have to modify your behaviour”. He told me that I should not swear: apparently I used the word “fuck” twice in an hour-long panel discussion. Why shouldn’t I swear? Perhaps because it is a sign of a poor vocabulary and/or low intelligence, and yet no such correlations have ever been found. Swearing is only ever harmful when it is used in an aggressive or abusive way, and not when used as a means of conveying excitement and emphasis. In these circumstances, the evidence shows that it does more good than harm, so the idea that swearing is bad is a fucking stupid one.

Overall UK happiness levels given English boost, ONS says Read more

But I digress. This man was insistent that, in my role as a professor at the London School of Economics and Political Science, I ought to be setting a “better” example to those who looked up to me. In doing so, he was appealing to a social narrative that places the burden on me to act in a particular way based upon my middle-class profession.

This social narrative and others like it have withstood generations of change. They have been shaped by power structures, cultures, laws, families, the media, historical practice and even evolutionary advantage. As well as satisfying some of our innate desires, such narratives have developed rules of thought and action that help to make a complicated world easier to navigate. By looking to the narrative for clues about how to live, we are provided with a coherent path to follow. Not only do we want to fit in, but we can also, like my festival friend, get angry with those who do not.

His behaviour and his reaction to me demonstrate just one way in which these stories can harm us and the people around us. They become what I refer to as narrative traps, which together form the myth of the perfect life.

The stories around wealth and success, in particular, are social narratives that we can’t seem to get enough of. Now, it should be obvious that the absence of either of these two things can cause anxiety and misery. I will not suggest otherwise. The narratives suggest, however, that no matter how much we have of each, we are expected to be reaching for more. The assumption is that ever more happiness is achieved with ever more money and more markers of success. The trap comes from the fact that the happiness hit from adherence to these narratives gets ever smaller the further up the ladder you go and, eventually, can become reversed. To be happier we need to move from a culture of “more please” to one of “just enough”.

Facebook Twitter Pinterest Illustration: James Melaugh

According to the UK’s Office for National Statistics (ONS), which has been studying the happiness levels of a sample of 200,000 people each year from 2011, about 1% of us are miserable. This would scale up to about half a million Britons. Earning less than £400 per week (or about £20,000 a year) is one of the factors that increases the chances of being in the most miserable 1%. Above £400 per week, the law of diminishing marginal returns kicks in. Once your basic needs are satisfied, your desire for ever-increasing amounts of money generates ever-decreasing returns of happiness.

Likewise, the most recent American Time Use Survey (ATUS), which allows analysts to estimate levels of happiness associated with a range of daily activities, showed that happiness goes up with increases in income at the lower end of the scale, but then it falls with higher incomes. Contrary to what most of us might predict, those earning over $100K are no happier than those with incomes of less than $25K. Those with the highest incomes report the least sense of purpose in their experiences. Perhaps “having it all” makes what we do feel less meaningful.

Data suggest that being rich can lead to time and attention being directed towards activities that fuel the attainment of more wealth, such as longer working hours and longer commutes, and away from activities that generate more happiness, such as time outside and time with family and friends. This discrepancy between the big effect on happiness that we imagine increased wealth should bring and the small effect we experience goes a long way towards explaining the narrative trap of reaching for wealth. But most people, including those on incomes above £50,000, truly believe misery would continue to fall with higher income above this point. And most people, irrespective of income, would continue to reach for more long after they have earned their 50 grand. This is the addiction problem.

Happiness is a goal worth pursuing – but it needs a feminist reboot | Stuart Jeffries Read more

If you are not struggling to make ends meet, I propose that you rein in the social narrative that encourages you to endlessly pursue more money. Invest your time and effort into doing all you can to ensure that those who are struggling are provided with the living conditions, wages and financial support that will help them to cover the costs of their living expenses. (Helping other people is great for our own happiness.)

A just-enough approach to wealth is not made any easier by the demands placed on us by family commitments, especially as your family grows, and social expectations. Social media, in particular, facilitate showing off beyond our wildest dreams. Even without the constant bombardment to reach for more wealth, “just enough” can still sound like a weak counter-narrative to “keep striving for more”. But even if it appears boring to accept that you (may) have enough wealth already, it can also be tremendously liberating. Once you have enough money to afford the basic things you want in life, you can stop constantly worrying.

Attending to being wealthy also means that we harshly judge others for being happy with what they have – we might call them unambitious or lazy – thus preserving the status quo and making it more likely that more people will be miserable with what they have. So we need to stop judging others as lazy, uninspiring or unambitious when they report being happy as they are. The narrative of reaching for wealth stigmatises those who do not want more money. Let’s instead celebrate those who choose to devote their time and effort to causes of social worth rather than question them for not accumulating more personal wealth.

The desire for wealth has truly far-reaching consequences. It usually means the overconsumption of goods, resulting in global greenhouse gas emissions and unnecessary land, material and water use. Repeated spending on items that are easily replaced means more production and excess waste, both of which have serious environmental consequences.

If you are a parent, reinforcing the narrative at home that some money can be enough will help children learn from an early age that the relentless pursuit of money is not inevitable. If you are a policy-maker, perhaps you could start by publishing lists of the top taxpayers, rather than the top earners. When I searched for “the world’s richest person” on Google, the answer came up immediately (Jeff Bezos was giving Bill Gates a run for his money when I last looked). But when I looked up “the world’s highest tax- payer”, the search returned lots of information on which countries had the highest tax rates. If we know that we are hardwired to compete and to compare, then let’s use this to write social narratives that we can all benefit from.

When it comes to the success narrative, the first box to tick is to be employed. But beyond having a job – any job – one of the most widely used measures of success is having a good job, and doing well in your career. In Happiness By Design, I told this story: A few weeks ago, I went out for dinner with one of my best friends, whom I have known for a long time. She works for a prestigious media company and basically spent the whole evening describing how miserable she was at work; she variously moaned about her boss, her colleagues, and her commute. At the end of dinner, and without a hint of irony, she said: “Of course, I love working at MediaLand.”

This story highlights the very common inner conflict between the social narrative of success, which values status and recognition in a job, and personal experiences of happiness in the job. My friend was experiencing pain and pointlessness at work, but the narrative she told about her job was totally unrelated. A job that makes us miserable is not a good job, but we can convince ourselves it is if it has high status. MediaLand is somewhere my friend had always wanted to work, her parents were proud of her, and her friends were a little bit jealous. So the narrative she created for herself comes from the broader social narrative of status.

The narrative surrounding status suggests that being a lawyer is a “better” job than being a florist. The latter is lacking in economic status and the former has plenty. But the MediaLand story also reminds us of another way in which one job might be “better” than another: namely, how happy it makes them day to day. And it is here that florists seem to have better jobs than lawyers, with 87% of florists agreeing that they are happy compared to 64% of lawyers. (This data comes from a 2012 City and Guilds survey that interviewed 2,200 employees from a wide range of professions; there was a 2013 follow-up for millennials that found pretty much the same thing.)

More recent data also suggests that the most conventionally “successful” occupations are not ones where the happiest workers are to be found. In 2014, the Legatum Institute published a report that looked to see which occupational groups were paid the most and which ones had the highest average life satisfaction. Predictably, chief executives and other senior officials were the highest paid, but they were no more satisfied than their secretaries, who were obviously paid much less. Some other professions whose members were happier than their bank accounts might suggest were the clergy, farmers and fitness instructors.

It could well be that those who choose careers like floristry or fitness might be happier to begin with than those who choose to go into law. We need good longitudinal studies (which follow the same people over time) to find out more. We might expect that many of those choosing careers as lawyers care more about what others think of them than those who choose careers analogous to being “just a florist”. However, there are aspects of jobs like floristry that make them more likely to generate happiness than working in a law firm. These include working with nature, regularly seeing the fruits of your labour, generally being around people who want to be with you, and feeling as though you have control over your workload. More than four out of five florists say that they are able to hone their skills every day, which makes them feel happy. Focusing on the likely daily experiences of the jobs we choose to take can help us avoid the unnecessary pain and pointlessness that often accompany adherence to the narrative of what a “good” job looks like.

The success narrative not only applies to what jobs we have but also to how long we spend working. It suggests that we should work longer and longer hours so that we can be wealthier and more “successful”. As incomes rise, it seems that we pay more attention to the income forgone from not working; and so we work more to capitalise on the increased value of our time. Time is money. Moreover, paying attention to time as money has been shown to diminish the pleasure experienced from leisure activities. Little wonder, then, that daily happiness in the US is actually lower for those on high incomes compared to those on middling ones. There is no time for enjoyment when you are using all of your time reaching to be rich.

When we look again at the ATUS, happiness and sense of purpose are both at their highest among people working between 21 and 30 hours a week, and misery increases in tandem with the number of hours worked thereafter. The results are consistent across genders. Many people do also choose to work long hours. Some people love their work so much that they want to spend as much time working as possible. I have felt like that at times in my career, and most of all when I have been working on my books, and I know many of my colleagues and collaborators have, too. But this is a quite rare and extremely fortunate position to be in.

Many more people “choose” to work long hours because the social narrative of long work hours is so persuasive. Most unpaid overtime (and perhaps some paid overtime, too) is undertaken out of a desire to progress at work and not from any underlying pleasure or purpose from working those hours. The expectation of prolonged working hours permeates a number of different professions, from banking, advertising and law to education and other public services, as well as poorly paid positions in the arts. There is huge pressure on staff to be the first one in and the last to leave, so everyone comes in earlier and leaves later.

Last year, I was involved in a TV series called Make Or Break? for Channel 5. A typical filming day lasted about 16 hours, from leaving the hotel to returning, and this was for six days out of seven, over a period of four weeks. Now, I’m not asking you to feel sorry for me, because being a TV presenter is hardly the worst job in the world (although the four-hour daily round trip in a van across dirt tracks to sit around in the heat of the Mexican summer waiting to film was about as bad as it gets for an impatient git who melts in temperatures above 20°C). The long days were justified on the grounds that this is “just how TV is”, as if historical precedent and industry expectations were explanation enough for overworking people. Long hours in TV are seen as a badge of honour, when instead they adversely affect happiness and, I suspect, productivity too.

As for the social narratives surrounding love and marriage, the narrative traps are everywhere. Think back on your bedtime stories as a child and I bet these words are lodged somewhere in your brain: “ …and they fell in love, got married, and lived happily ever after.” These imagined happy endings stick with us as adults. An overwhelming majority of us report considering marriage as part of our ideal lifestyle and we often project this preference on to others, too. An unmarried 40-year-old is “unlucky” or has yet to meet “the one”: as if being married is something for all of us, and that there is some one – one person – out there for every one of us.

This is wilfully overoptimistic. Any given relationship is much more likely to end than it is to result in living happily ever after. Two in five marriages in the UK end in divorce. We have to come to terms with those odds. We expect passionate love to be nothing but pleasurable (when, in fact, it can be quite damaging, just like any other compulsion) and to last for ever (when, for the vast majority of us, the high dies after about a year), and our spouse to satisfy all of our needs (which no one human being ever can). Against this background, as Esther Perel, a psychotherapist who has written a lot about marriage, has asked: “Is it any wonder that so many relationships crumble under the weight of it all?”

When a relationship ends, particularly a long-term one, so many people are heard to say “what a shame” or “such a waste”. But if for the most part during that relationship both partners were happy, how can it be a shame or a waste of time? A breakup is most likely to be in the best long-term interests of both parties. How many of you are knowingly with someone now who is worse than a previous partner? We are highly adaptive creatures who are very good at moving on. So, if in doubt, best get out. Don’t let the narrative fool you into staying past the point it would be better for you to leave.

Divorce hurts the children of couples who separate, though, right? Well, yes, to a degree. Researchers from the University of Virginia have shown that children of divorced parents do experience negative emotions, including anxiety, shock and anger, in the short run; however, for the vast majority of kids, these feelings have dissipated within a couple of years of their parents’ divorce. Those born into high-conflict relationships report greater happiness as adults if their parents split up compared to those whose parents stayed together. So, on balance, it’s probably better to be the child of divorced parents than to be the child of parents who stayed together and who argue a lot. The social story of staying together for the sake of the children potentially harms the children more than accepting when a relationship has gone wrong and walking away from it in a way that preserves the kids’ happiness. We should perhaps start using the word “congratulations” for divorce, as well as for marriage.

There are many decision-makers who could start to tackle the narrative traps of love and marriage. Parents could warn their children about the dangerous fairytale narrative that love always means living happily ever after. This would then result in less psychological, financial and health-related support being required for the eventual fallout from a less than perfect marriage. Schools could do more to provide teenagers with the basic facts on love: that you can expect the passion to diminish would be a good place to start. This will help youngsters to see through the fog of social narratives so that they can make more informed choices.

The legal system also needs to rethink how it deals with marriage and divorce, and the welfare of children affected by both. There are good libertarian and egalitarian grounds for the state to stop incentivising marriage (eg, by removing the tax breaks for married people that are not available to single people). It can, instead, allow for a range of different contracts between two or more individuals, so that people can specify the rights and responsibilities in the relationship that best suit their unique set of circumstances. Where the state does intervene in family matters, it should focus directly on strengthening the relationships that each parent has with their children. In this way, governments can focus on what really matters – strong parenthood to the benefit of children – without having to interfere in the relationship between the parents.

Going against the grain of social narratives can be challenging. As I said earlier, it is likely that the task is being made harder by the powerful role played by social media, which, overall, have made visible, and consequently magnified, the importance of having achieved according to the main narratives.

It seems glib yet still important to say that social media are not the problem, it’s just that we don’t know how to control our consumption. And it’s not all doom and gloom: social media might also lead to the formation and reinforcement of groups that are anti-narrative. You can find Becoming Minimalist, a group with more than 800,000 members, which actively celebrates spending less money with an emphasis on sustainable living. And then there’s The Childfree Choice group on Facebook with a following of more than 20,000, which allows people who decide against having kids to share experiences and promote the benefits. This can act as a nice counterbalance to the prevailing narratives.

Returning to the guy who accosted me at that festival at Hay-on-Wye, he was very clear that I had a duty to act in a certain way. As far as he was concerned, I was “playing the working-class hero” in a distasteful way. Judging by some of the comments that have been written about me online, he is not alone. One of my favourites, for what it’s worth, was “Maybe happiness can be found in the wearing of unpleasant spectacles? And chavvy watches” in response to a photo of me in the gym, wearing my white specs and a sports watch, for a piece I wrote for the Guardian.

When I stated publicly a while ago that I do not like to read fiction, I was judged very harshly for this by other academics as well as in the press. So not only am I expected to conform to stories about how LSE professors ought to behave when I am working, I am expected to use my leisure time in ways that conform with the stereotype, too. I have often felt obliged to rein in who I am because I do not conform to what is expected of me in my career. It rarely feels pleasurable to go against the grain, but it often feels purposeful. But it shouldn’t always feel so bloody hard.

I would like those of you who feel more of an obligation to fit in, at work or elsewhere, to not be chastised if you really would like to behave in ways that are not consistent with the prevailing social stories. Ultimately, insofar as we have any choice over such matters, each of us has to decide for ourselves when we conform and the circumstances under which we want to stand out. We can each live our lives in ways that reduce our own misery as much as possible, properly accounting for the impact that our actions have on other people.

We can make decisions on behalf of others according to the same misery-minimising rules. We should have more respect than we currently do for those who forsake social narratives that simply aren’t for them. Here’s to us all escaping the myth of the perfect life, and to being Happy Ever After.

• This is an edited extract from Happy Ever After: Escaping the Myth of the Perfect Life by Paul Dolan, published on 17 January by Allen Lane, £20. To order a copy for £17.60 go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99

About the author

Facebook Twitter Pinterest Paul Dolan at the London School of Economics. Photograph: Jeremy Baile

Paul Dolan is professor of behavioural science at the London School of Economics. He is an expert on the measurement of happiness and acts as a wellbeing adviser to the government and a number of NGOs. His first book, Happiness By Design (2014), which argued that finding a sense of purpose as well as pleasure in life makes us happier, became a bestseller January, looks at how social narratives – “the stories about how we ought to to live our lives” – can hurt us as much as they may help.

Born in 1968, Dolan was raised in east London and Essex and was the first person in his family to go to university. He studied economics at Swansea before moving into the field of behavioural science at the University of York. A former visiting scholar with Professor Daniel Kahneman, the bestselling author of Thinking, Fast and Slow, at Princeton university, Dolan held posts at the universities of York, Newcastle, Sheffield and Imperial College London before joining the LSE.

When asked in 2014 what makes him happy, he said: “Happiness is situated in what we do and who we spend time with. It does not reside in some story we tell ourselves about what we think should make us happy.”

Dolan says he finds happiness in his work, going to the gym, evenings out with friends, having new experiences, talking to taxi drivers… What makes him unhappy? Other people’s weddings, reading novels and overlong family holidays, he says. He concludes: “I am a very lucky man: not because I have a great job and family and all that stuff but because I have a sunny disposition. Now that makes me happy.”