I’ve been called many things in my life, but never an optimist. That was fine by me. I believed pessimists lived in a constant state of pleasant surprise: if you always expected the worst, things generally turned out better than you imagined. The only real problem with pessimism, I figured, was that too much of it could accidentally turn you into an optimist.

But accidental optimism is not one of the known dangers of pessimism, a list that does include career impairment, poor health and early death. Optimism, by contrast, is associated with better sleep and lower levels of cardiovascular disease. One study this year claimed that people who describe themselves as optimists had 35% fewer strokes than those who didn’t. Another, published in the Proceedings of the National Academy of Science last summer, found that compared with pessimists, the most optimistic subjects lived 11-15% longer lives on average.

One of that study’s authors, clinical research psychologist Lewina Lee, described optimism as “a psycho-social asset that has the potential to extend the human lifespan”. She also suggested that a more positive outlook could be learned. “Interestingly, optimism may be modifiable using relatively simple techniques or therapies,” she said. But how does a pessimist go about transforming himself into the glass-half-full type? I decided to spend a week trying.

Day one

Before I go in search of the bright side, I need to make sure I actually count as a pessimist. I had always considered myself more of a realist, someone who sees the glass as comprised of approximately 50% water. What’s wrong with that? “A certain amount of stoicism is a good idea to help you cope with disappointment,” says the author and psychotherapist Philippa Perry. “The trouble is, if you are always looking at the ditch instead of the road ahead, you tend to end up in the ditch instead of where you wanted to go.”

But being relentlessly optimistic sounds as problematic as being a full-time pessimist – both demonstrate a poor understanding of how odds work. Routinely expecting good outcomes just seems wrong. “We tend to confuse the familiar with what is right,” says Perry.

Maybe I am more optimistic than I think, but I don’t even know how you would define optimism for scientific purposes, much less measure it. There are, it transpires, two main types: dispositional optimism and the “explanatory style”. Dispositional optimism can be described as the sense that more good things than bad things will happen in the future; explanatory or attributional optimism is more concerned with the causes behind good events, and the extent to which we view them as controllable and stable.

Dispositional optimism is commonly quantified using a questionnaire called the Life Orientation Test – Revised (LOT-R). It is one of the measures used in Lee’s study and versions of it are widely available online. The LOT-R consists of a series of statements (eg “I rarely count on good things happening to me”), that are rated on a five-point scale running from “strongly agree” to “strongly disagree”. I always think tests like this just measure my tendency to fence-sit – I don’t “strongly agree” with anything. But the results were unequivocal: I scored nine out of a possible 24, with the average somewhere between 14 and 15. “You are quite pessimistic,” is the diagnosis.

‘Quite pessimistic’ … Dowling. Composite: Jill Mead

Day two

When it comes to improving my outlook, it is hard to know where to begin. I contact Lee to find out how scientists do it. “The Best Possible Self (BPS) intervention is the most widely studied training for increasing optimism,” she says. As Lee explains it, the exercise involves identifying some personal, professional and relationship goals, and then imagining a future in which those goals have been reached and everything has turned out as well as possible. In one study, participants were asked to turn this BPS into a coherent and personal story, and then imagine this story for five minutes every day over two weeks.

I find a version of the exercise online that encourages me to write continuously about my future BPS for 15 minutes. This is what I end up with:

The first thing I write in my gratitude journal is: ‘This is never going to work’

“I have become a bestselling novelist, and I am so much in demand as a public speaker that I now employ someone to read and answer my emails. He is called Charles and even though this is his only duty, I pay him an absurdly generous salary, because that is what I am like. Thanks to my extreme generosity, Charles has come to regard me as foolish and out of touch. He has recently started to steal small amounts of money from me, along with some of the free luxury items I am often sent as a result of being a celebrated writer, which Charles thinks I will not miss.

“He is right. I am not really interested in unsolicited luxury items, since I have everything I want already: an enviable career, rippling abdominal muscles, a special room for my awards. And I never have to spend a second reading or responding to emails. Every morning I wake up feeling excited about the challenges that lie ahead, and every night I go to bed grateful that the only real source of anxiety in my life is the upcoming talk I need to have with Charles about him taking the piss.”

I’m not supposed to worry about taking steps to achieve this goal, which is good, because I can’t think of any.

Day three

One of the simplest strategies for increasing optimism is avoiding the company of other pessimistic people. I figure that I have a headstart here, in that I already avoid the company of most people.

The doorbell rings. I think: this can’t be good. Then I think: stop that. The man at the door has a package for me. My wife passes through the kitchen as I’m opening it.

“What’s that?” she says.

“It’s my gratitude journal,” I say, holding up a slim notebook with the words “Start with gratitude” written on the cover in a self-helpy calligraphic font.

“Stupid,” my wife says.

“If you’re not going to be positive about my journey,” I say, “then you and I might have to stop hanging out.”

“That can be arranged,” she says.

I don’t disagree with her; the idea of a gratitude journal sounds stupid to me, too. If I dropped dead tomorrow, I wouldn’t want anyone finding it among my effects. But Perry is adamant about the benefits of the practice. “This is scientifically proven stuff. It’s fun, really,” she says.

Life coach and psychologist Dr Sally Ann Law offers similar advice. “Develop a habit of making a note at the end of each day of at least three things that went well,” she says, “to help train your brain to notice the good stuff rather than focusing on the not-so-good stuff.”

To overcome my resistance to the idea, I have ordered a ready-made “five-minute daily” journal with blanks to fill in. At the beginning there is a lined page headed: “My Thoughts”. I write: “This is never going to work.”

Day four

I spend five minutes thinking about my Best Possible Self, putting off my difficult talk with Charles for another day. In my gratitude journal, under the blank space headed “I am thankful for …” I write: “A small child in a pushchair said hello to me in the street yesterday, and I said hello back.” It is only a small thing, but it made me happy at the time and it makes me happy again thinking about it. Without the gratitude journal, I would have probably forgotten about the moment. I am pleased, but I can’t believe I have got to come up with another one tomorrow.

There are endless online articles compiling strategies to increase optimism. One of the odder ones involves giving my pessimism a name – something such as “Mr Negative” – so that I can catch myself falling into habitually disempowering patterns. I tell myself there is no way I’m naming my pessimism, before realising I already have: Charles.

Other advice includes the acquisition of a positive mantra – something inspirational I can repeat to drive out pessimistic thoughts. My gratitude journal has similar epigrams at the top of each page: “Every day may not be good but there is something good in every day.”

Another one, attributed to the entrepreneur Mary Kay Ash, catches my eye: “Aerodynamically the bumblebee shouldn’t be able to fly, but the bumblebee doesn’t know that so it goes on flying anyway.” I am afraid I seized on this one for pessimistic reasons: it sounds like bollocks, and some casual research proves it is. Under the space in my gratitude journal headed “Things I want to remember from today …”, I write: “Bumblebee flight does not violate the laws of physics.” I think: progress.

But here is another inspirational quote I found – it comes from Michael Jordan, perhaps the greatest basketball player of all time: “I’ve missed more than 9,000 shots in my career. I’ve lost almost 300 games. Twenty-six times I’ve been trusted to take the game-winning shot and missed. I’ve failed over and over and over again in my life. And that is why I succeed.”

Those lines seem tailor-made for the reluctant optimist, because they encapsulate something counterintuitive about outlook: we tend to think of pessimists as being better prepared for setbacks, but in truth it is the optimists who prove more resilient.

“Optimists tend to interpret adversity as being a discreet and temporary state,” says Law. “Pessimists, however, tend to interpret adversity as being pervasive and permanent: ‘It’s all really horrible and that’s probably just the way it will always be.’”

This has an effect on more than just your outlook. Studies have shown that optimistic students spend more time preparing for exams than their pessimistic counterparts.

I copy out the Jordan quote into my gratitude journal under “Nice things I heard today …” I feel silly doing it, but it uses up a big chunk of the page.

Looking on the bright side … Composite: Jill Mead

Day five

I try to spend five minutes reflecting on my BPS, but the whole scenario is getting away from me. I imagine that Charles has taken to hiding my things in weird places, and then claiming that I put them there and forgot about it. When I finally sit him down for our stern talk about taking the piss, he changes the subject and asks me to co-invest in his podcast start-up. I tell him I’ll think about it.

In my journal, under the heading “Things I did for myself today …”, I write about spending an hour tidying my desk for the first time in months, but I don’t mention I was basically avoiding other, more pressing work. I feel I as if I am covering for my own failure and then trying to get credit for it by writing it down. That could describe my whole career.

I re-read a strategy that Law emailed me at the beginning of the week. “Become as self-aware as possible about the language you use in your own head when you’re thinking about some adversity or setback,” she wrote, suggesting that when I catch myself using words such as “disaster” and “dreadful”, I should replace them with words such as “challenge” and “unhelpful”.

I visualise finding my desk tidy in the morning, and how pleased with myself I will feel. Later, in the middle of cooking dinner, I discover that I have failed to buy a key ingredient. “Well,” I say, staring into the fridge, “this is a complete and absolute challenge.” Baby steps.

There is no way I’n naming my pessimism. Then I realise I have – his name is Charles

Day six

I sleep badly and wake up with a familiar sense of foreboding. Visualising a Best Possible Self strikes me as a waste of time, a way of deliberately courting disappointment. Besides, I am avoiding Charles.

By the afternoon I am in a dark and self-defeating mood. In my journal, under the heading “I am grateful for …”, I have absentmindedly scrawled: “Get cat food.” I think about an article from the publication Trends in Cognitive Sciences I read that suggested dispositional optimism is at least partly inherited – to some degree, you either have it or you don’t. And I don’t.

Law suggests I should not expect too much from a mere week of accentuating the positive. “Remember that although our brains are plastic and therefore we can change to some extent, we have to be realistic,” she says.

Perry concurs. “Changing a thought reflex or a pattern is a bit like going to the gym,” she says. “You don’t go to the gym once and expect to leave with a six-pack, but if you go regularly and keep it up, changes do happen.”

After I actually go and buy the cat food, I feel a bit better.

Day seven

There can hardly be a less scientific measure for increased optimism than a second go at the LOT-R test. It has got only 10 questions, four of which are fillers (eg, “I enjoy my friends a lot”) that don’t count toward the result. It can easily be gamed. But I didn’t expect my score to change at all and it did: I got 14, which is about average. I was conscious of no difference beyond thinking about my answers more carefully, in the context of what I had learned.

Perhaps, for the pessimist, greater self-awareness leads inexorably to optimism. As Perry put it: “When we become more aware of our typical response, we can recognise what we are in the habit of telling ourselves. When we are aware of a habit, we then have a choice about whether to change it or not.” Once you accept that at least a portion of your pessimism is just a story you tell yourself – and an inaccurate one at that – it’s hard to hang on to it.

With that in mind, it is time to sit down and have a little talk with Charles.