In a modern capitalist economy, almost everything is for sale, including risks. Markets can transfer known risks to people or institutions who can handle the risk more effectively. The cost of rebuilding my house if it is destroyed by fire is beyond my means. For an insurance company with many properties on its books and many shareholders, by contrast, the loss is easily manageable.

Trade in risk may be motivated by such insurance—the prudent management of risk—but it may also be about wagering: that is, individuals backing their own, distinctive, perception of the nature and likelihood of an event. People trade risk because they see the same risk differently. I think Arsenal will win the match, but you favour Chelsea. So we take bets on the outcome, and one of us will win and the other will lose. We have divergent opinions, or information, or we believe we do. In the long run, of course, it is only the bookmaker, or the house, that wins.

Financial markets accommodate both prudent insurers and reckless gamblers. They provide investors with an opportunity to diversify their portfolios, and allow gamblers to bet on future movements in interest rates. The coexistence of the two can allow speculators to make profits by stabilising prices—buying when markets are fearful, and selling when they are greedy. But when the gambling motive overwhelms the insurance motive, speculation becomes destabilising and then risk, far from being minimised by careful management, becomes concentrated in the hands of those who understand least what they are doing. And when regulators perceive insurance when they should see wagering, their actions magnify a crisis rather than minimise it. Such destabilising speculation, mischaracterised by regulatory authorities as prudent risk assessment, is what caused the global financial crisis of 2008.

The coexistence of insurance and gambling goes back to the earliest days of markets in risk, and the interaction of the two has been central to financial history. But it was four developments in the second half of the 17th century that combined to frame the way we think about risk, and the institutions we have for dealing with it, through to the present day.

Coffee is thought to originate in Arabia, and was introduced to Europe in the 16th century. Coffee houses were found in Venice before 1650, but the Grand Café on Oxford High Street, which opened in 1654 (and is still a coffee shop today) was the first in England. The fashion spread: English gentlemen gathered in coffee shops to talk and do business—and to wager.

The era saw a resurgence of gambling. Humans have played games of chance since the beginning of history—primitive dice, made from animal bones, have been found at prehistoric sites. But public policy towards gambling has regularly swung from prohibition to liberalisation and back, as regulatory regimes do. The restoration of the monarchy in 1660 ended Puritan restrictions on gambling and much else, and the pendulum swung decisively towards permissiveness. It has done so again in the last 50 years.

Another development was in the understanding of probability. Historians trace the beginnings of such analysis to a correspondence between two great French mathematicians, Blaise Pascal and Pierre de Fermat, in the winter of 1654. The correspondence was supposedly prompted by a conversation between Pascal and the Chevalier de Méré, an aristocrat with some flair for the card table. It may seem strange that the discovery of probability came so late in the history of thought. After all, the ancient Greeks were no mean mathematicians, and they gambled: and the mathematics of probability is not, as mathematics goes, especially difficult. But before the 17th century there was little concept of probability in the modern sense. Fortune represented the will of the gods, and could not be described by the kinds of laws appropriate for the natural world. The word “probable” shares the same root as “approve,” and “probable” would mean “accepted by the best authorities.” Even in the 18th century, Edward Gibbon would write of Hannibal’s crossing of the Alps, “the account of Livy is the more probable,” by which he meant widely acknowledged and approved, “but that of Polybius is the more true.” The development of probabilistic thinking depended on the Enlightenment’s rejection of argument from authority and the diminished influence of religious doctrine.

And modern insurance also traces its roots to the second half of the 17th century. Contracts with some of the character of insurance policies have a long history, but the Great Fire of London in 1666 led directly to the establishment of the first insurance business, the Insurance Office, in the following year. The Hand in Hand Insurance Company, founded in 1696, is thought to be the oldest surviving insurer: it is today part of Aviva. These offices didn’t just pay up when there was a claim, as a modern insurer does: they sent an engine to try to put the fire out. On some London buildings you can still see fire marks, the means by which insurers, or their operatives, could recognise the property of their policyholders. Such competition was not a very efficient system: the different private fire services established agreements for mutual assistance, but it was not until the 19th century that London enjoyed a public fire service, funded from taxation.

The coffee shops of London, named for their proprietors, provided a venue for these new businesses. Tom’s coffee shop was an early centre of insurance. Jonathan’s was a venue for securities trading, and is regarded as the genesis of the London Stock Exchange. White’s and Brooks’s, the first London clubs for gentlemen, were founded as gaming houses.

The most famous coffee shop of all was that of Edward Lloyd. Gentlemen met there to wager on the weather, the tide and the fate of ships at sea. Merchants realised that they could use this facility to lay off part of the risk of overseas trade. In due course that coffee shop became the insurance market we know today as Lloyd’s of London. The inter-related histories of gambling, insurance and actuarial mathematics began in coffee shops in the 17th century. It was this mix of entertainment, business and sophisticated analysis which would—some three and a half centuries on—reach a climax in the global financial crisis of 2007-8.

The first insurance policies were fire and marine, but the scope of the industry extended to life insurance with the development of actuarial science, the gathering of accurate statistics from which life expectancy could be assessed. In the mid-17th century, John Graunt had constructed the first mortality table by inspecting burial records. But it was not until the following century that the ultimately ill-fated Equitable Life Assurance Society was founded, “equitable” because its premiums were calculated on actuarial principles. In contrast, in the coffee houses, gentlemen with little interest in mathematics took bets on when the king would die, and whether Admiral John Byng would be executed.

Aside from being distasteful, this wagering was dangerous. How would you react if your neighbour bought insurance on your life, or against your house burning down? In 1768, the London Chronicle thundered that “when policies come to be opened on two of the first peers in Britain losing their heads and on the dissolution of the present parliament within one year… it is surely time for the administration to interfere.”

The administration did indeed interfere. The courts established the doctrine of uberrima fides—an insurance contract was void unless the insured had declared everything that might be relevant to the other party’s assessment of the risk. And legislation created the concept of “insurable interest,” whereby you could take out insurance only if you would suffer some demonstrable loss, financial or emotional, from the event against which you sought to insure; thus saving you from the temptation of assassinating your neighbour in order to benefit from insurance. The law sought to draw a line between insurance, based on economic interest, and wagering for amusement. It was important, and a natural consequence, that insurance contracts were legally binding, but wagers were not. An English gambling debt was a debt in honour only, enforceable only by social convention—or private force. Such provision remained part of English law until 2005—only then did bets at William Hill become legally binding.

The 1892 case of Carlill vs Carbolic Smoke Ball Company is a favourite of law teachers, because the proceedings, while essentially ludicrous, illustrate some key points of English law. The Carbolic Smoke Ball Company made extravagant claims, and in particular promised that its smoke ball would provide protection from influenza. The company was so confident that it offered £100 to anyone who caught “flu after using the product as instructed.” Louisa Carlill purchased a Carbolic Smoke Ball for 10 shillings—no mean sum in those days—and claimed to have used it three times a day as instructed; until she contracted influenza, at which point she claimed the £100. Two letters were ignored by the company, but on the third request the Carlills did receive a response. The company requested that they attend their office to use the machine and be checked over by the secretary. At this point, they took the company to court.

The barrister for the company was the young Herbert Asquith, later to become prime minister, who advanced an ingenious range of arguments in the company’s defence. One such argument was that the promise of £100 was a wager, and hence the claim was unenforceable.

But Henry Hawkins, the judge and a keen racegoer, recognised gambling when he saw it. He defined a wager for the purposes of English law: “A contract by which two persons, professing to hold opposite views touching the issue of a future uncertain event, mutually agree that dependent on the determination of that event one shall win from the other.” I have not been able to establish whether Mrs Carlill ever received her £100.

As the case was making its way through the courts, the applied mathematics of probability was developing in leaps and bounds. The theory of statistical inference, which provides the mathematical underpinning for quality appraisal in production processes, for the increasingly controversial judgments about public opinion based on polls of voters, and for clinical trials of new pharmaceutical projects, dates from that era. Probabilistic mathematics found a new field of influence in the 20th century as quantum mechanics disapplied the deterministic laws of classical physics to things too small to be visible.

Meanwhile, economists struggled to explain behaviour towards risk with their theory of rational choice. How could the same individual appear to embrace risk at the racetrack but shun it when the home insurance policy came up for renewal? In 1948, Milton Friedman and his Chicago colleague L “Jimmie” Savage proffered an answer to that apparent inconsistency. They postulated—in effect—that money initially gave steadily decreasing rewards as basic needs were met, with each extra pound or dollar bringing less and less psychological benefit. But then, at some higher income, each additional pound or dollar would yield greater rewards once again; perhaps, for example, because once some threshold was reached, extra resources might facilitate a leap into a different lifestyle.

It all sounds a bit contrived, and it was: the explanation has more ingenuity than plausibility. A recent survey by Daniel Friedman (no relation) and colleagues concluded: “We toured 60 years of empirical search and conclude that no such functions have yet been found that are useful for out-of-sample prediction,” and furthermore that the same theories had no application in making sense of “major risk-based industries such as finance, insurance and gambling.” In short, the Friedman-Savage analysis explained only the particular behaviour it had been designed to explain, and was inconsistent with much observed behaviour.

But for a majority of economists, any explanation will do if it enables them to maintain their insistence that behaviour corresponds to their own subject-specific concept of “rationality.” And that assumption of rationality yields the intellectual framework that has governed both practice and supervision in financial markets. It is a framework that acknowledged no differentiation between risk and gambling—both were assumed to be manifestations of rational choice under uncertainty, based on risk appetite. And central bankers and other regulators adhered to this misconception through to the global financial crisis—and beyond.

But even as economists conflated insurance and gambling, the legal distinction between gambling and insurance remained. The case of Carlill vs Carbolic Smoke Ball Company and the concept of insurable interest became important again when the Credit Default Swap was invented in the 1990s. This is a security which commits the institution underwriting the “swap” to make a payment in the event of default on a specified loan. If you, as the purchaser of a CDS, had made the loan yourself, you were buying insurance. If you hadn’t—a “naked” swap—you were simply betting that the borrower would fail to repay. The original issuer of the first credit default swaps was usually an insurance company—the best-known provider was the London-based Financial Products Group of AIG, America’s largest insurance company: but anyone could issue, buy or sell such securities, on any loan, and might want to do so whether or not the buyer or seller had a relevant economic interest.

If a credit default swap was an insurance policy, it would have to be taxed and regulated under the insurance regime. If it was a wager, it would have been liable to gaming tax and regulation and exempt from ordinary contract law. The International Swaps and Derivatives Association sought a legal opinion on the status of credit default swaps from Robin Potts, a leading London barrister. Potts found that credit default swaps were neither insurance nor wagers—exactly the opinion the Association had hoped for, because it left them in an ungoverned legal limbo. This advice—not legally binding but widely accepted—allowed London to become a centre for trading credit default swaps. Not to be outdone, in the United States, President Bill Clinton signed the Commodity Futures Moderation Act into law in December 2000, which took most derivatives out of the scope of securities regulation altogether.

When the Gambling Act received Royal Assent in April 2005, part 17 of that Act repealed English statutes dating back to 1710, and set out for the first time that “the fact that a contract relates to gambling shall not prevent its enforcement.” By then, the City of London was doing a roaring trade among gamblers.

Meanwhile, over in the mountain resort of Jackson Hole, in Wyoming, the Federal Reserve Bank of Kansas was in the midst of its annual symposium which attracts many luminaries of the financial world. The 2005 conference was especially memorable because its purpose was to celebrate “the Greenspan era.” Alan Greenspan’s 20-year tenure as Chairman of the Federal Reserve Board was to come to an end the following year. During his term of office, the market for credit-related derivatives, such as CDSs, had grown explosively—the total amounts expressed in such derivatives exceeded the value of all the assets in the world.

But was it gambling, or insurance? One attendee, Raghuram Rajan, the former chief economist at the International Monetary Fund, was sceptical about all these new instruments. He described what the market was doing as the acceptance of “long-tail risk”—bets against events which had a low probability of occurrence but involved large financial loss if they did occur. Many of these long tail risks were not independent but related to each other, potentially triggered by the same sort of crises. But if they ever materialised simultaneously, the potential for crisis was evident.

Joseph Cassano, the former CEO of the AIG Financial Products Group, one of the principal underwriters of CDSs, would describe what he was doing in very similar terms—but with a crucial twist. He was happy to assume risks that he believed would simply never materialise. Even in 2007, he said “it is hard for us, and without being flippant, to even see a scenario within any realm of reason that would see us losing $1 in any of those transactions.”

Rajan’s take on the brave new world was not well received. Ex-Fed Governor, Don Kohn, explained that Rajan had mistakenly identified insurance as gambling. “By allowing institutions to diversify risk, to choose their risk profiles more precisely, and to improve the management of the risks they took on, they have made institutions more robust… these developments have also made the financial system more resilient and flexible—better able to absorb shock without increasing the effect of such shocks on the real economy,” Kohn explained. Larry Summers, the former Harvard academic and Treasury Secretary under President Clinton was less restrained—he denounced Rajan’s paper as “Luddite.”

If the difference between gambling and insurance still needed elaboration, it came from the events that followed the 2005 Jackson Hole conference. One such event was more immediate even than the 2008 collapse of AIG Financial Products and the $85bn bailout of its parent company by the US government. As the participants climbed into jets and limos to return to their offices, Hurricane Katrina blew into New Orleans, killing at least 1,800 people and imposing the most costly single group of losses in the history of the global insurance industry.

And the following year, the US hedge fund manager John Paulson and others would place what the author Michael Lewis would call, in his book of the same name, the Big Short—the bet against US sub-prime mortgages. Paulson used “synthetic collateralised debt obligations”—securities whose value depended on a package of subprime mortgages he had identified as particularly likely to fail. He then bought credit default swaps linked to these obligations. Paulson was making a bet against the US housing market and needed someone else to take up the other side of his wager. As a Goldman Sachs salesman for these swaps described “the Abacus transaction”—to his girlfriend, though not to his clients: “I had some input into the creation of this product which by the way is a product of pure intellectual masturbation, the type of thing which you invent telling yourself: ‘Well, what if we created a “thing,” which has no purpose, which is absolutely conceptual and highly theoretical and which nobody knows how to price?’”

It is not hard to distinguish insurance against a real flood from the sale of dangerous and synthetic credit related derivatives.

The outcome is history. The US market for sub-prime mortgages went into meltdown in 2007-8. Paulson and the other heroes of The Big Short won at the expense of the clients of Goldman and the other investment banks who had taken the opposite side of his wager. The losers were rescued by central banks (and Cassano was forced to retire from AIG, though given a $1m a month consultancy contract, later terminated after public outrage).

In 2010, senior Goldman Sachs executives appeared before a Senate Committee to give evidence on the Abacus transaction. The outcome was a famous exchange. Senator Carl Levin, a Democrat, asked: “When you heard that your employees in these emails and looking at these deals said ‘God, what a shitty deal,’ ‘God, what a piece of crap,’ when you hear your own employees or read about these in emails, do you feel anything?” David Viniar, the chief financial officer at Goldman Sachs, replied: “I think that is very unfortunate to have on email.” It was all far distant from the world of uberrima fides—or that of insurable interest.

Most people reading this history conclude that more regulation of both gambling and insurance is necessary. Yet one observation should give pause for thought. Potts was right when he concluded that early CDSs were not wagers. These instruments originated instead in “regulatory arbitrage,” the construction of instruments which have the same economic effect as a regulated transaction but receive more lenient treatment or entirely avoid regulation. Banks were required to provide capital in relation to the size of the loan; insurance companies by reference to the scale of the expected loss. If a loan to ExxonMobil, large in amount but entailing very low risk, could be treated as insurance rather than borrowing, the required capital provision would be much smaller.

It would only be later that the financial sector would turn the credit default swap into an instrument which threatened the stability of the global financial system. Blythe Masters, the JP Morgan executive who is credited with the invention of the credit default swap, and Potts, who did not appreciate the momentous consequences of his opinion, could never have imagined that their work would become central to a Hollywood movie. And yet one of the greatest of all Hollywood movies, more than half a century earlier, had previewed the moral and intellectual confusion behind the global financial crisis and The Big Short. The benignly corrupt French police officer who closes Rick’s Bar in Casablanca explains:

Captain Renault: “I’m shocked, shocked to find that gambling is going on in here!”

a croupier hands Renault a pile of money

Croupier: “Your winnings, sir.”

Captain Renault: [sotto voce] “Oh, thank you very much.”

[aloud] “Everybody out at once!”

But it would not be long before Rick’s Bar reopened, its customers only marginally chastened by their experience.