About 40 minutes into the new movie Deepwater Horizon, there is a scene in which crew member Jimmy Harrell is suddenly called into the dining area on the floating drilling platform of the same name. “Mr. Jimmy,” played by Kurt Russell, was the offshore installation manager (essentially the crew boss) on the vessel, and he’d been summoned to receive a special award for the rig’s excellent safety record. Like most of the scenes in the movie, this one is closely based on actual events the night of April 20, 2010, when the actual Deepwater Horizon was destroyed by an uncontrolled eruption of oil and gas. The explosion killed 11 crew members and set off the worst oil spill in U.S. history.

In fact, the Deepwater rig had gone an extraordinary seven years without a single accident serious enough to halt operations. The rig, owned by the Switzerland-based Transocean Ltd., and its veteran crew were some of the best in the business. (BP, the world’s sixth largest oil company at the time, was leasing the rig, a bit the way a rich sportsman might a charter fishing boat and crew.) Just months earlier, BP and the Deepwater team had broken the record for deepest well ever completed. Yet, even as Harrell was being handed his award, high-pressure oil and gas were threatening to surge up the pipe from the sea floor. Despite all their experience and advanced technology, the crew members didn’t spot the signs of trouble and, once the blowout started, didn’t act quickly enough to contain it or save the rig.

Like all Hollywood versions of actual events, Deepwater Horizon takes a few liberties with the facts. But that scene accurately captures the central paradox of most large man-made disasters: How could such well-trained experts make decisions that, in retrospect, appear so deeply flawed? Why didn’t they see the disaster coming or stop it in time? The film paints a gripping picture of a technological catastrophe, and it showcases some genuine heroism on the part of several crew members. But to answer the question of how the disaster happened in the first place, we need to dig a little deeper.

The Dutch pilot-psychologist Sidney Dekker, one of the pioneers in studying large technological breakdowns, has written that the true cause of most disasters is not so much the initial accident “but the failure to identify the accident early in its birth.” The blowout of BP’s Macondo Prospect well was a case study in how a series of small mistakes and misjudgments, when not caught in time, can snowball into catastrophe.

The movie places the blame squarely on the BP executives who helped direct the drilling operations. John Malkovich, playing Don Vidrine, BP’s “company man” on the rig, fairly drips malice as he pushes the crew to cut corners. In a gumbo-thick Louisiana drawl, he berates the Transocean men for being “nervous as cats.” The reality is more complex: BP consistently made some decisions that favored speed over safety, and the company had a reputation for being particularly hard-driving. But the Transocean crew was also involved in the dubious decision-making. And the federal regulators who supervised drilling in the Gulf of Mexico signed off on their plans at every stage.

The reality is that both BP and Transocean had grown dangerously overconfident and were pushing too close to the edge. Perhaps overly impressed by the team’s good safety record, federal regulators routinely rubber-stamped the BP/Transocean proposals. Moreover, despite claims to the contrary, none of the drilling companies in the Gulf had a workable scheme to cope with a massive oil spill. The entire industry had succumbed to risk creep: Over the decades, drillers gradually moved into deeper waters and sunk wells that involved much greater internal pressures and hazards. The technologies and regulations originally developed for shallow waters were updated in response, but not to a degree commensurate with the growing risks. So, even as drillers were getting more proficient, disaster was becoming more, not less, likely.

The Transocean crew, the BP executives, the federal regulators—none of these were stupid people. BP may have had incentives to push its drilling teams hard, but even the greediest executive knows there’s no upside to a catastrophe that kills people, causes massive ecological damage, and costs the company tens of billions of dollars. Malkovich’s scenery chewing notwithstanding, BP’s company man Vidrine certainly didn’t expect that his decisions that day would lead to him to be nearly incinerated by midnight. There has to be a better explanation for why intelligent people sometimes make such terrible decisions.

And there is. After the loss of the space shuttle Challenger in 1986, sociologist Diane Vaughan began a long investigation into the accident. Her findings would challenge many of our easy assumptions about how disasters occur. We like to think that accidents happen because bad people knowingly and carelessly let them happen. Vaughan discovered something more troubling: that even organizations staffed by smart, seemingly moral people can slowly slide into dangerous and unethical behavior.

Vaughan, an expert in corporate malfeasance, wanted to know how NASA officials made the decision to launch the Challenger despite a serious last-minute safety concern. Very cold weather was forecast for launch day, and some engineers worried that the low temperatures might worsen a long-standing problem: The shuttle’s solid-fuel booster rockets had a tendency to leak small jets of hot gas during takeoff. The engineers urged a delay. NASA decided to launch anyway. The standard view of the accident holds that NASA brass overruled the nervous engineers out of concerns that allowing yet another launch delay would hurt NASA’s image with the public and Congress. From this view, the managers knowingly rolled the dice, bending the safety rules in order to stay on schedule.

Vaughan spent nine years researching the question and determined just the opposite. In her monumental book, The Challenger Launch Decision, Vaughan demonstrates that NASA officials rigorously followed their own safety guidelines.

“Managers were, in fact, quite moral and rule abiding as they calculated risk,” she writes. And yet they were flat wrong. “Following rules, doing their jobs,” she concludes, “they made a disastrous decision.” The managers weren’t “amoral calculators,” Vaughan says. Instead, they were fundamentally deluded about the risks posed by the leaky boosters. What’s more, over the years they had systematically deluded themselves, through a process she calls the “normalization of deviance.”

Here’s how the normalization of deviance works: Early in the shuttle program, the appearance of small leaks from the booster rockets’ rubber seals was an unexpected and alarming event. NASA assigned a working group, which dutifully studied the issue and determined the leaks would be manageable as long as they didn’t exceed a certain threshold. “They redefined evidence that deviated from an acceptable standard so that it became the standard,” Vaughan writes. Sure enough, small booster-seal leaks were soon seen as routine during shuttle launches. The problem had been normalized. But as shuttle missions continued, the leaks kept getting bigger. Each time, NASA repeated the process, again determining that the seal failures were acceptable as long as it didn’t exceed certain, ever higher, thresholds. NASA had crept right to the edge of what would cause a mission failure, all the while convinced that it was operating safely. The fact that the shuttles kept flying reinforced its false sense of security. Then came something NASA hadn’t anticipated: a launch day so cold that it made the rubber seals hard and brittle. The huge resulting leak burned a hole in the shuttle’s external fuel tank.

Vaughan’s normalization of deviance theory had a big influence on the growing branch of management studies dedicated to preventing disasters. Business and engineering schools started teaching the concept. (Vaughan herself now teaches at Columbia University.) Vaughan and other researchers argue that most high-risk industries are prone to normalizing deviance. We’ve all seen this, even in businesses that don’t involve life-and-death decisions: Managers focus on positive data about their operations and tune out small signs of trouble, safety margins get shaved in the name of efficiency, and small deviations from procedural rules are tolerated. But disaster researchers have also developed strategies to help counteract the tendency: tools to help managers be more aware of “weak signals” hinting at trouble, for example, and policies that empower whistleblowers. Companies that follow these and related safety strategies are known as high-reliability organizations—a concept with which BP and Transocean executives should have been deeply familiar.

But the evidence shows that the Deepwater Horizon rig was anything but a high-reliability organization. The movie realistically captures some of this dysfunction. In one scene, we see that the computer used by the rig’s chief electronics technician (played by Mark Wahlberg) is often on the fritz. In another, we learn that an office smoke detector is broken. In fact, many alarm systems on the rig were deliberately “inhibited” in order to prevent false alarms from waking up the crew. On the sea floor, a crucial structure of pipes and valves known as the blowout preventer was poorly maintained (and probably not robust enough in the first place). The blowout preventer was supposed to be the last-ditch defense against high-pressure gas and oil bursting out of the well. It failed utterly.

In designing the structures that would stabilize the pipe and prevent leaks below the sea floor, BP repeatedly opted for the quickest, rather than the most secure, approaches. Though some choices were debated, there’s little sign that the drillers saw any of these decisions as necessarily or blatantly dangerous. (And the regulators certainly didn’t object.) Collectively, though, those aggressive choices nudged the whole operation toward higher risk. BP and Transocean had been working this way for a while. When the wells didn’t fail, the drillers—just like those NASA officials—saw that as a vindication of their methods. Even as their wells were getting more dangerous, they grew more confident and complacent. Researchers who study disasters tell us that a long period without an accident can be a big risk factor in itself: Workers learn to expect safe operation as the norm and can’t even conceive of a devastating failure. In such a situation, workers and managers with the most experience are often the last to recognize when risks are getting out of control.

After every big disaster we naturally assume some reckless decision by a manager or a massive equipment breakdown must have been the cause. In the words of accident investigator Dekker, we look for “bad people, bad decisions, broken parts.” But in studying large industrial failures (including the BP spill), Dekker came to a very different conclusion about what causes such problems. Large accidents are more often the result of dozens of tiny contributing factors: misguided assumptions on the part of workers and managers; small, subtly flawed decisions; routine mechanical or digital glitches. Individually, none of these seem particularly noteworthy to the people on the front line—just another day on the job. It’s only after the accident that we see how this particular row of dominos toppled. What Dekker, Vaughan, and others conclude is that large accidents are primarily the result of particular workplace cultures. In Vaughan’s words, “Mistake, mishap, and disaster are socially organized and systematically produced by social structures.”

This new social science of disaster helps explain why the people involved never seem to see the accident coming. The people working in these cultures don’t think they are being reckless; they don’t recognize all the ways they’ve normalized deviance and let risks creep up. When the disaster finally strikes, they are as stunned as anyone. These findings also show why disasters are so hard to predict: According to Dekker, “Accidents can happen without anything breaking, without anybody erring, without anybody violating the rules they consider relevant.” The disaster, in other words, is not a violation of the daily routine, but a product of it. Some disaster researchers call these sorts of incidents “normal accidents” or “organizational accidents” to stress the way they emerge from the normal operation of the organization.

BP’s Macondo Prospect blowout was a textbook case of an organizational accident. One scene in the movie shows the crew conducting two critical pressure tests to check the well’s integrity. The first test produced a confusing and worrisome result. So they ran a second test, using a different pipe, and got the result they wanted. (Forensic analysis later hinted that the pipe used in the second test may have been clogged.) The first test must have been wrong, they concluded. Everything was fine. The team had succumbed to the normalization of deviance, setting aside data that didn’t conform to their expectations and relying on information that did.

The movie presents the discussion over the test results as a showdown between BP’s oleaginous Vidrine, pushing to ignore the bad test, and Transocean’s stalwart Harrell, urging caution. In reality, the confusion over the test results was shared, and the decision to move ahead despite ambiguous data was apparently not unusual. Of course, it’s hard to fault the filmmakers for amping up the venality of the BP execs onscreen. Movies need villains (and BP is hardly in a position to expect sympathetic treatment). But if we come away from the movie believing that disasters such as the Gulf spill are caused primarily by flamboyantly greedy and reckless executives, we are learning the wrong message. The kinds of decisions that lead to disasters are rarely telegraphed by malevolent executives twirling their moustaches. More often, they appear just like any other decision in a busy workday.

The behavior of the Deepwater Horizon crew also indicated that none of them was particularly alarmed that work was proceeding despite the worrisome test. Over the next few hours, the crew overlooked several signs that pressure was growing in the well. When the explosive surge of gas and oil finally reached the drilling rig, Harrell was off taking a shower. The blowout caught the rest of the crew off guard as well. It took the crew on the bridge more than a minute to sound the general alarm and much longer to hit the disconnect button that would separate the rig from the well. By then it was too late. This doesn’t mean the crew members were bad at their jobs—it means they were human. In any complex business the expectation of normal operation runs deep. The same workplace culture that gradually normalizes risk-taking also makes it hard for workers to expect trouble. In many disasters, the front-line workers struggle even to comprehend what is happening; the unfolding events fall completely outside their mental models.

To say that the Gulf oil spill was an organizational accident—something that grew organically out of the culture of the drilling platform—doesn’t mean BP wasn’t responsible. It was BP’s decision to attempt extraordinarily deep and hazardous drilling operations. And BP would reap enormous profits from a successful well. So the company deserves to pay every penny of the more than $60 billion in fines and settlements it has been charged with. Moreover, companies that operate in very hazardous fields have a heightened responsibility to understand modern thinking about how workplace cultures can incubate disasters. The various strategies that make up the concept of the high-reliability organization are not obscure or impossible to implement. It is hard to imagine a work environment more in need of those principles than an oil platform. An analysis by University of California–Berkeley’s Center for Catastrophic Risk Management concluded: “This disaster was preventable if existing progressive guidelines and practices been followed.” Instead, BP and Transocean doubled down on their aggressive style, convinced that their past successes were proof of their skill and invulnerability. In the words of the Berkeley group, “They forgot to be afraid.”

So, by all means, BP deserves to be excoriated. And the Deepwater Horizon movie misses few opportunities on that front. (As one critic wrote, if “you need a corporate stooge who helped destroy the Gulf of Mexico to come off as absolutely evil, yes, you will want to hire John Malkovich.”) But if we think that we can prevent future accidents simply by pointing fingers at the people who caused the last one and exonerating ourselves as less corrupt, we are fooling ourselves. Preventing disasters requires a special kind of daily vigilance that needs to be actively taught and constantly reinforced. Only by admitting that, yes, it could happen to us too, can we take the steps to make disaster less likely.