In 2011, the New York University psychologist Gabriele Oettingen published the results of an elegant study, conducted with her colleague Heather Kappes, in which participants were deprived of water. Some of these parched volunteers were then taken through a guided visualisation exercise, in which they were asked to picture an icy glass of water, the very thing they presumably craved. Afterwards, by measuring everyone’s blood pressure, Oettingen discovered that the exercise had drained people’s energy levels, and made them relax. The implication is startling: picturing an imaginary glass of water might make people less motivated to get up and head to the watercooler or the tap in order to quench their real, non-imaginary thirst.

This conclusion is precisely the reverse of one of the central tenets of pop psychology: the idea that picturing the future you desire makes it more likely you’ll attain it. Again and again, in her research, Oettingen has shown that making a fantasy of something you want can make it harder to achieve in reality. Imagine yourself having a productive week, and you’ll accomplish less. Imagine receiving a windfall of cash, and you’ll be less motivated to engage in the kinds of activities that might bring you money. Intriguingly – though admittedly the link may not be causal – there’s even a relationship between how much “positive thinking” language American presidents use in their inaugural speeches, and how much unemployment rates change by the end of their presidential terms. The more positive the fantasy about the future, the fewer jobs in real life.

Fist-pumping motivational gurus have long claimed that your brain “can’t distinguish between reality and imagination”. Ironically, Oettingen’s experiments show they’re right about that – but also that the conclusion they draw is spectacularly wrong. Attempting to “experience your success as if it had already materialized” is a fast-track to disappointment.

Thankfully, not all kinds of thinking about the future are quite so self-sabotaging. In Oettingen’s new book, Rethinking Positive Thinking, published in the US on Thursday and elsewhere next month, she makes the case for “mental contrasting”, a technique that involves methodically combining positive and negative thoughts about your own future, in a way that seems to work strikingly well if you’re trying to replace bad habits or mindsets with good ones. Over the last decade, in studies conducted by Oettingen and other researchers, mental contrasting has been shown to double the amount of exercise people engage in; to increase their fruit and vegetable intake by 30%; and to help people suffering from chronic pain become more physically active. When low-income schoolchildren in Germany and the US were trained briefly in the method, it led to increased school attendance and better academic performance.

The technique’s full formal name is less than catchy – “mental contrasting with implementation intentions” – so in her book, Oettingen rebrands it as “Woop”, for “wish, outcome, obstacle, plan”. The acronym sets out the four stages of the process. First, spend a minute or two thinking in detail about something you wish to accomplish; second, vividly imagine the best thing you associate with having achieved that outcome. (That “best thing” might be an emotion, a promotion, praise – anything, really.) Third, ask yourself what internal obstacle’s most likely to get in the way. (This isn’t about your boss, or your spouse, so much as that weakness inside you that holds you back from better pay or a better relationship.) Finally, formulate an “if-then” plan for what you’ll do when that obstacle arises. (“If I find myself feeling terrified when I stand up in front of the audience, then I’ll recall how diligently I’ve rehearsed.” “If I find myself checking Twitter, I’ll get up from my desk immediately.”)

There’s nothing wrong with a bit of positive daydreaming if it makes you feel good, so long as you don’t expect anything more than feeling good. “Daydreaming can be great!” Oettingen told me the other day. “It’s only when it comes to actually realizing these dreams that positive dreaming is problematic, because it feigns that you’ve already attained these desired futures. You relax, your energy goes down. It’s a question of: ‘Why face the hardships of the bumpy road to achieving these things in real life, if you can float away on dreams?’”

Mental contrasting retains the most useful part of positive fantasies: it clarifies what you want, and reminds you how good it can feel to attain, say, a pay increase, a new job, a smaller waistline or a happier relationship. But then it exploits the motivating power of knowing what you have not yet attained – that there’s serious ground you’ll have to cover. (Getting fit, improving a marriage, making more money – it all takes work.) Finally, the Wooping technique nudges you to plan in advance for what you’ll do once, inevitably, your own flawed personality gets in the way.

And so mental contrasting pushes back against one of the great, disempowering lies of positive thinking: that transformative change, personal or societal, can be made effortless. It can’t. In fact, Oettingen’s research suggests, convincing yourself that life’s meant to be easy just makes it appreciably harder. The best way to quench your thirst for change, it seems, is to remind yourself that the glass is still half empty.