Legendary Dutch footballer Johan Cruyff used to slap his goalkeeper in the stomach before each match. Tennis ace Serena Williams always bounces her ball five times before her first serve. Jennifer Aniston, it is reported, touches the outside of any plane she flies in with her right foot before boarding.

From touching wood for good luck, to walking around ladders to avoid bad luck, we all have little routines or superstitions, which make little sense when you stop to think about them. And they are not always done to bring us luck. I wait until just after the kettle has boiled to pour the water for a cup of tea, rather than pouring just before it boils. I do not know why I feel the need to do this, I am sure it cannot make a difference to the drink.

So, why do I and others repeat these curious habits? Behind the seemingly irrational acts of kettle boiling, ball bouncing or stomach slapping lies something that tells us about what makes animals succeed in their continuing evolutionary struggles.

Repeat behaviour

We refer to something that we do without thinking as being a habit. This is precisely why habits are useful – they do not take up mental effort. Our brains have mechanisms for acquiring new routines, and part of what makes us, and other creatures successful is the ability to create these habits.

Even pigeons can develop superstitious habits, as psychologist B. F. Skinner famously showed in an experiment. Skinner would begin a lecture by placing a pigeon in a cage with an automatic feeder that delivered a food pellet every 15 seconds. At the start of the lecture Skinner would let the audience observe the ordinary, passive behaviour of the pigeon, before covering the box. After fifty minutes he would uncover the box and show that different pigeons developed different behaviours. One bird would be turning counter clockwise three times before looking in the food basket, another would be thrusting its head into the top left corner. In other words, all pigeons struck upon some particular ritual that they would do over and over again.

Skinner's explanation for this strange behaviour is as straightforward as it is ingenious. Although we know the food is delivered regardless of the pigeon's behaviour, the pigeon doesn't know this. So imagine yourself in the position of the pigeon; your brain knows very little about the world of men, or cages, or automatic food dispensers. You strut around your cage for a while, you decide to turn counter clockwise three times, and right at that moment some food appears. What should you do to make that happen again? The obvious answer is that you should repeat what you have just been doing. You repeat that action and – lo! – it works, food arrives.

From this seed, argued Skinner, superstition develops. Superstitions take over behaviour because our brains try and repeat whatever actions precede success, even if we cannot see how they have had their influence. Faced with the choice of figuring out how the world works and calculating the best outcome (which is the sensible rational thing to do), or repeating whatever you did last time before something good happened, we are far more likely to choose the latter. Or to put it another way: “if it ain’t broke, don’t fix it”, regardless of the cause.

Habit forming

University of Cambridge psychologist Tony Dickinson has taken the investigation of habits one step further. Dickinson trains rats to press a lever for food and perform another action (usually pulling a chain) for water. The animals can now decide which reward they would like most. If you give them water before the experiment they press the lever for food, if you give them food beforehand they pull the chain for water.

But something strange happens if the animals keep practising these actions beyond the point at which they have effectively learnt them - they seem to “forget” about the specific effects of each action. After this “overtraining”, you feed the animal food before the experiment and they keep on pressing the lever to produce food, regardless of the fact that they have just been fed. The rat has developed a habit, something it does just because it the opportunity is there, without thinking about the outcome.

Sound like anyone we know? To a psychologist, lots of human rituals look a lot like the automatic behaviours developed by Skinner's pigeons or Dickinson's rats. Chunks of behaviour that do not truly have an effect on the world, but which get stuck in our repertoire of actions.

And when the stakes are high – such as with sports – there is even more pressure on our brains to “capture” whatever behaviours might be important for success. Some rituals can help a sportsperson to relax and get “in the zone” as part of a well-established routine before and during a big game. But some of the habits you see put my kettle boiling routine to shame. Tiger Woods always wears red the last day of a golf tournament, because he says it is his “power colour”. In baseball, Wade Boggs claimed he hit better if he ate chicken the night before. Soccer’s Kolo Toure once missed the start of the second half because refused to come out – superstition dictated he had to be the last player to re-emerge from the dressing room, but on that occasion he was stuck there waiting for a stricken teammate to finish treatment.

We cling to these habits because we – or ancient animal parts of our brains – do not want to risk finding out what happens if we change. The rituals survive despite seeming irrational because they are coded in parts of our brains, which are designed by evolution not to think about reasons. They just repeat what seemed to work last time. This explains why having personal rituals is a normal part of being human. It is part of our inheritance as intelligent animals, a strategy that works in the long-term, even though it clearly does not make sense for every individual act.

If you would like to comment on this story or anything else you have seen on Future, head over to our Facebook page or message us on Twitter.