After a long history of problems, two river outlet valves deep inside Oroville Dam were not operating at a critical time this winter, eliminating one option to help manage severe storm runoff.

California water officials had another tool at their disposal to help empty Lake Oroville before damages to the spillway led to massive evacuations downstream. But it was out of commission despite years of repair work.

That tool is the river outlets in Oroville Dam: two massive pipes that draw water from deep in the reservoir, through the dam and directly into the Feather River.

The river outlets have been intermittently functional ever since an accident inside the dam on July 22, 2009. Five workers were injured that day when they were ordered to operate the river outlets in a manner that ignored prior warnings by engineering experts.

The outlets, officially known as the river valve outlet system (RVOS), are an important fourth option to move water out of the reservoir, in addition to the powerhouse and two spillways.

“Apparently there are still some engineering concerns about those river outlets,” said Ronald Stork, a senior policy advocate at Friends of the River in Sacramento, an expert on flood-control operations at major dams. “They need river outlets that work, because this might happen again.”

In January and February, a series of wet atmospheric river storms pounded the Feather River watershed upstream of Oroville Dam. The resulting heavy runoff caused water to rise rapidly behind the dam, forcing the California Department of Water Resources (DWR), which owns and operates the dam, to release massive flows from the dam’s main spillway.

For reasons that remain unclear, the concrete spillway began to break apart, and DWR opted to close the spillway gates. The reservoir continued to fill until water began flowing over the emergency spillway, which is nothing more than a concrete lip emptying onto a dirt hillside.

DWR assumed the emergency spillway would never have to be used, and dismissed earlier warnings that it was a poor design. But soon erosion of the hillside threatened to undermine the spillway lip, which could have caused a catastrophic and uncontrolled release of water. DWR reopened the main spillway gates to prevent that, causing further damage to the spillway and additional erosion into the riverbed.

Meanwhile, nearly 200,000 people living downstream were forced to evacuate as a precaution.

It is for just such reasons that dams are built with multiple water outlets: Redundancy ensures safety.

At Oroville, however, redundancy evaporated: The river valves were out of commission; the main spillway was ripped apart, causing erosion that dammed the river channel. This prevented water releases from the dam’s Hyatt hydroelectric power plant, which was flooded by the rising water.

Officials at DWR say the river outlets are operational again. But in early January, when the storms rolled in, the outlets had been taken out of service while fire safety improvements were carried out in the hydropower plant, said DWR spokesman Ted Thomas.

The river outlets are two long pipelines, each 35 feet in diameter, that run from the reservoir bottom on the upstream side of the dam to the base of the dam on the other side.

The outlets date to the earliest days of Oroville Dam’s construction in the mid-1960s. Their first purpose was to divert the Feather River itself around the dam construction site. The dam was built on top of the outlets, which then became an important low-level drain once the dam was finished.

One of the pipes was plugged with concrete when the dam was completed. The other holds two smaller pipes, each 6 feet in diameter, that are controlled by valves.

Today those two pipes serve as a backup means to pass water through the dam if Hyatt Powerplant is out of commission for any reason. Each helps release water from the reservoir, if necessary, before it fills to the elevation of the main spillway.

Because the river outlets are also the lowest drain in the dam, they have become important for accessing the coldest water at the bottom of reservoir to sustain endangered salmon runs. But they have rarely been used for that purpose until recently.

The 2009 accident occurred when five DWR employees were directed to descend into the river valve outlet chamber deep inside the dam, and open the valves to 100 percent as a maintenance test.

The turbulence caused by the massive water flow through the valves caused hurricane-force winds inside the tunnel, ripping a steel bulkhead from its mountings and blasting the workers off their feet. They were pinned against equipment inside the tunnel, holding on for survival until other workers were able to shut off the massive water flow. All survived, but one suffered a broken leg and arm.

A subsequent investigation found that DWR supervisors ignored prior advice from U.C. Davis engineering experts, who warned in 1993 that the valves should never be opened to 100 percent. That’s because an “energy dispersion baffle ring” lining the tunnel, meant to reduce turbulence, had been damaged in 1968 when the dam was completed.

DWR removed the baffle ring in April 2009, four months before the test. It did so without consulting its Division of Dam Safety, which should have approved that action first. The test was ordered because DWR “wanted to determine the destructive effects” of opening the valves to 100 percent, according to the investigation.

The California Occupational Safety and Health Administration subsequently fined DWR for a number of safety violations.

Three years passed before a settlement was reached with Cal/OSHA in 2012 allowing DWR to begin repairing the river valves. This included a new hydraulic system and hardware allowing the valves to be operated remotely, so workers don’t have to stand in the tunnel to open them.

California’s severe drought finally brought action to fix the river valves. DWR and its State Water Project customers, who ultimately pay for any repairs to the dam, realized in 2014 that water levels in Lake Oroville could drop so low that the river valves would become the only way to get water out of the reservoir.

If the outlets couldn’t operate, DWR would violate numerous water quality and endangered species regulations.

So in a span of about four months in 2014, DWR spent $11 million testing and installing new river valves. Afterward, it operated the valves at limited capacity for several months in 2014 and 2015 to release water into the Feather River.

Full release capacity at the valves was not restored until 2016, when DWR installed a new baffle ring in the river outlet tunnel. To date, Thomas said, DWR has spent $18 million refurbishing the river valves.

Thomas said the valves can now release water at their full design capacity, which is 5,400 cubic feet per second when the reservoir is full.

That wouldn’t have been enough, he said, to take strain off the dam’s main spillway when it began to crumble in February – or to prevent use of the spillway in the first place – because inflow was so much greater. At times, total storm runoff into the reservoir exceeded 100,000 cfs.

Michael Johnson, a professor of engineering at Utah State University who has studied the dam, agreed.

“They have a pretty minor amount of flow contribution,” Johnson said. “The use of those two valves is really for releases when they cannot generate hydropower, or when the reservoir gets too low.”

Never miss an update. Sign up here for our Water Deeply newsletter to receive weekly updates, special reports and featured insights on one of the most critical issues of our time.