Statisticians tell us that the chances of winning the lottery are incredibly small – for the UK National Lottery, for example, around one in 14 million per ticket. That’s about the same probability as seeing a flipped fair coin come up 24 heads in a row, and far less than your chances of being killed by a meteorite. And yet, week after week, people do win, supplying newspapers with a constant flow of personal-interest stories into the bargain. What’s going on? How can something that has such an incredibly small chance keep on happening?

The explanation is, of course, straightforward. The chance that your ticket is the winner is indeed small. But you’re not the only person entering the draw. In fact, many people buy lottery tickets each week. Often, they buy more than one. So, overall, a very large number of tickets are bought. And while each ticket might have a very small chance of winning, if we add up that very large number of very small chances, it soon amounts to something sizeable. With enough people buying enough tickets, we should actively expect to see someone win.

This distinction – between the chance that you (or, indeed, any other particular person) will win the lottery and that someone will win – is a manifestation of what I call the law of truly large numbers. If a large enough number of people each buy a lottery ticket, then the probability that someone will win becomes substantial. It grows so large, indeed, that someone wins almost every week.

This law is part of what I have called ‘the improbability principle’. The principle states that extremely improbable occurrences are, in spite of the odds against them, actually quite common. It says we should expect to see events that we might regard as incredibly unlikely – such as someone winning the lottery. The improbability principle consists of five elements, of which the law of truly large numbers is just one.

Allow me to introduce its partners in – not crime exactly, but… Well. You’ll see.

The law of inevitability says that some outcome must occur – one of the 14 million sets of six numbers from one to 49 must be chosen when the lottery balls drop. So, if you bought all possible combinations, you’d be certain to hold the jackpot-winning ticket. That sounds trivial but, of course, people have still found a way to make money out of it.

The law of selection says, in effect, that while prediction might be hard, postdiction is easy. It’s easy to look back and see the causal chain that led inexorably to disaster. It’s not so easy to choose among the multitude of possible chains that lead into the future.

The law of near enough says that you can dramatically increase the chances of a coincidence if you broaden what you mean by a coincidence. You would be surprised to encounter an old friend in a strange town, perhaps, but you might be almost as surprised if you met a friend of a friend, even though friends of friends heavily outnumber friends.

Finally, the law of the probability lever says that slight changes can make highly improbable events almost certain. Thus we encounter financial crashes, positive results in ESP experiments, people being repeatedly struck by lightning and so on.

Or take the RMS Titanic. This flagship of the White Star Line had a double-bottom hull to make sure that the chances of water flooding in it were very small. Furthermore, it was divided into 16 different compartments using bulkheads with remotely operated watertight doors. For the ship to sink, several of the compartments would have to flood simultaneously. And if the probability of one compartment flooding was very small, then surely the probability of several doing so would be a great deal smaller.

For these reasons, many people regarded the ship as unsinkable. And the basic line of reasoning seems, on the face of it, pretty compelling: just as the probability of you winning the lottery is very small, so the probability of you winning it several times is much smaller. If you buy one ticket on the UK National Lottery, your chance of winning is around one in 14 million. If you buy two tickets on consecutive weeks, your chance of winning both times is about 1 in 2×1014, or roughly the same chance as tossing a fair coin and seeing 48 heads in a row. In other words, don’t hold your breath.

And yet the Titanic did sink. Why? Well, there’s nothing wrong with the lottery calculations. If you win the lottery one week with a one-in-14-million chance per ticket, then your chances of winning it the next week are unaltered. Statisticians say that the two events are independent, but another way to put it is that the lottery numbers don’t remember who has won previously: the outcome of one draw doesn’t affect the following one. It follows that the probability that you will win the lottery in both weeks is just the two separate probabilities multiplied together: the 1 in 2×1014 mentioned earlier.

The same does not hold for the Titanic. For if one compartment is damaged so that it floods, what does that say about the probability that a neighbouring compartment might also be damaged? Well, clearly our answer depends how the damage occurs. As it happens, the Titanic’s maiden voyage was through iceberg-infested waters. If an iceberg were to strike the side of the ship penetrating the double hull, isn’t there a good chance that it would also damage neighbouring compartments?

even very slightly incorrect assumptions can have huge consequences

Icebergs can be very large – especially the part hidden beneath the water – and the ship would be moving past them. This means that the two events – damage to one compartment and damage to another – are not independent. And this, of course, is exactly what happened. The iceberg didn’t simply puncture one compartment and then bounce off. Rather, it cut into the side of the ship at several points, flooding six compartments.

What we find is that the appropriate way of thinking about what happened to the Titanic is different from the appropriate way of thinking about the lottery. We have to change our model slightly, relaxing the assumption that the events in question (different compartments flooding) are independent. The result is that what the ship’s owners and passengers believed was a highly improbable event in fact becomes quite likely.

I chose the Titanic example because it’s very clear and straightforward: it’s easy to see why the independence assumption is unjustified. In many cases, however, it’s not obvious which assumptions are incorrect – and even very slightly incorrect assumptions can have huge consequences, especially when they interact with other strands of the improbability principle, such as the law of truly large numbers. We live in a complex world, and the different components of a system are often locked in a web of interconnections that are difficult to tease apart. When trying to make sense of them, it is common to assume independence as a first approximation. But this can lead to major miscalculations. The Yale sociologist Charles Perrow has developed an entire theory of what he calls ‘normal accidents’, based on the observation that complex systems should be expected to have complex, undetected, interactions. A frightening thought.

But I should note that the probability lever doesn’t just have adverse consequences. Consider Joan Ginther, a woman in her sixties from Texas, who has won some $20 million from lotteries in four separate wins: $5.4 million in 1993 (though this ticket was bought by her father rather than Ginther herself); $2 million in 2006; $3 million in 2008; and most recently, $10 million in 2010. The first win was from a standard lottery, where you had to pick six numbers, but the other three were from scratchcards.

Now, any strand of the improbability principle can increase your chances of winning a lottery – and therefore your chances of winning more than once. Buying multiple tickets, for example, can bring the law of truly large numbers into play. Ginther is reported to have bought some 3,000 scratchcards per year, spending perhaps $1 million in total on them. A lot of tickets – a lot of chances of winning. But that’s still not enough to make her multiple wins very likely. We need to bring the probability lever into play.

The most familiar kind of lottery is known as an ‘r/s’ lottery, so called because each ticket consists of r numbers chosen from a list of possible s numbers. They are simple and well-understood. Scratchcard lotteries, on the other hand, are more complex – and this complexity provides a crack for the lever to enter.

she periodically bought large numbers of tickets in one go: it’s as if she had cracked the routing algorithm the company used to deliver the tickets

Suppose the Texas lottery operators just sent out all 3 million scratchcard tickets in one go. That would mean that all the winning tickets might end up getting bought very quickly, leaving no incentive for players to buy the remaining tickets. This, of course, would leave the lottery operators out of pocket. And so, instead of spreading the prize money at random over the tickets as they are printed and released, the operators try to make sure it is fairly uniformly distributed.

In fact, the 3 million tickets are released in six consecutive tranches of half a million each, with each tranche containing one-sixth of the prize money. The next tranche is released only when most of the tickets for the preceding one have been bought, which encourages people to keep playing. Analysis of data from the Texas Lottery even suggests that the algorithm keeps at least some of the big prizes for the later tranches, to keep the game interesting. If this is true, it means that the probability of winning one of the big prizes is not uniform – it isn’t the same whichever ticket you buy. And so we find an opening that the probability lever can exploit.

Knowing when large jackpot tickets are likely to be sold is half of the battle. But to be practically useful, you have also to have an idea of where – so that you can buy tickets there. Ginther bought three of her winning tickets in a small town called Bishop in Texas, where she was born, not far from the Mexican border. Although she moved to Las Vegas, she periodically returned to Bishop and bought large numbers of tickets in one go: it’s as if she had cracked the routing algorithm that the shipping company used to deliver the tickets. (For the record, it’s probably worth mentioning that Ginther has a PhD in mathematics from Stanford University and spent some years as a college lecturer in California.)

The story of Joan Ginther shows us one way in which the probability lever can gain purchase on the seemingly impregnable bulwarks of chance. But there are plenty of others. In fact, even some standard r/s lotteries have concealed structures that can serve as a pivot point.

Lotteries, of course, seek to make money for the people or organisations that run them. Their basic mechanism for doing this is to return only a percentage of the total amount paid for the tickets. This means that the expected return for a regular player is less than $1 for every dollar you bet. Players should expect, on average, to lose money. But lottery draws are repeated, week after week, and if the jackpot is not won in one week, it is typically ‘rolled over’ to the next. By buying lottery tickets only when the rollover jackpot has built up substantially, you can boost your expected winnings to more than your stake: then you can expect, on average, to make money.

That’s all very well, but the amount you expect to win in the long term if you keep playing and the chance that you will win it in the short term are two very different things. It is unrealistic to take a long-term perspective lasting thousands of years. What else can you do?

buying tickets only when things were in their favour, several groups made significant sums of money

You might draw inspiration from several groups in Massachusetts. The Massachusetts Cash WinFall was a 6/46 lottery, meaning that one had to choose six numbers from one to 46, with tickets being drawn twice a week. The jackpot began at $500,000 but there were other prizes – of $4000, $150, and $5 – for matching five, four, or three numbers respectively. Whereas many lotteries roll an unclaimed jackpot forward, adding it to the jackpot of the next draw, this lottery rolled it down if it exceeded $2 million without being won: so the lesser prizes, awarded for matching fewer than all six numbers, went up in value.

The Massachusetts groups spotted that if the rolldown exceeded a certain amount, then the total they would expect to win on the cumulated rolldown prizes would exceed what they would have to spend on tickets. Spotting this, and buying tickets only when things were in their favour, several groups made significant sums of money. So much so, in fact, that Cash Winfall was terminated at the start of 2012.

That’s the trouble with the probability lever: sometimes it moves mountains, and sometimes it breaks off in your hand. And of course, when it does, the law of selection means it’s difficult to find another one. Then again, the improbability principle just says that the improbable is commonplace: it’s another matter entirely whether or not you can summon it at will.