Since 2010, at least three ruptured pipelines have spilled oil into U.S. neighborhoods, forcing officials to decide quickly whether local residents would be harmed if they breathed the foul air. But because there are no clear federal guidelines saying if or when the public should be evacuated during an oil spill, health officials had to use a patchwork of scientific and regulatory data designed for other situations.

As a result, residents of the three communities received different levels of protection.

No houses were evacuated in Salt Lake City, Utah, where a ruptured pipeline leaked 33,000 gallons of medium grade crude oil before it was discovered on the morning of June 12, 2010. The oil ran down Red Butte Creek, past neighborhoods where windows were left open in the summer heat. The fumes, which are known to cause drowsiness, left some people so lethargic that they didn't wake up until after noon.

In Marshall, Mich. officials called for a voluntary evacuation after more than a million gallons of heavy Canadian crude spilled into the Kalamazoo River on July 25, 2010. But they agonized over the decision for four days before making that recommendation.

In Mayflower, Ark. authorities quickly evacuated 22 families after a broken pipeline leaked about 200,000 gallons of heavy crude on March 29, 2013. But people living in the same subdivision, just a few blocks away, were not asked to leave. Neither were the residents of the lakeside community where the oil eventually pooled and where the cleanup continues today.

After each of these spills, people complained of headaches, nausea and respiratory problems—short-term symptoms that health experts say are common after any chemical spill and usually disappear as the air clears.

What health experts don't know, however, is whether the fumes could also trigger long-term health problems that become evident only years or decades later. That gap will be increasingly important, because over the next few years the industry plans to build or expand more than 10,000 miles of oil pipelines—including the Keystone XL.

Many of these pipelines will go through or near populated areas. For instance, the Michigan pipeline that ruptured in 2010—Enbridge Inc.'s line 6B—is being replaced with a larger line that will pass so close to some homes that one family is losing part of its back deck.

But despite the pipeline boom, there are no plans to conduct long-term health studies in Mayflower, Marshall or Salt Lake City. There also doesn't appear to be any momentum to set federal guidelines for chemical exposures at oil spills, so health officials will be better equipped for future emergencies.

"The key question that people have—'Will I be affected 20 years later given my two-week exposure'—is something no one can answer," said Judi Krzyzanowski, an environmental consultant in Ontario, Canada who studies air pollution from oil and gas development. "If people in Mayflower develop cancer five years from now, it will be nearly impossible to point a finger at the oil spill."

Crude oil typically contains more than 1,000 chemicals, many of them hazardous to humans. Of particular concern is benzene. Small amounts of benzene from car exhaust and cigarette smoke are commonly found in the air. But increased exposure is known to cause leukemia and neurological problems.

Despite decades of research, it's difficult to determine exactly how much benzene is too much. Although the federal government offers dozens of guidelines for benzene concentrations in air and water, each comes with different caveats and none is designed for oil spills in residential neighborhoods.

The federal Agency for Toxic Substances and Disease Registry (ATSDR), for instance, estimates that people can be exposed to air containing 9 parts per billion (ppb) of benzene for up to two weeks, or 6 ppb for up to a year, without a "likely" increase in harmful health effects. But those guidelines don't cover the risk of cancer, and they are "not intended to define clean up or action levels for ATSDR or other Agencies," according to ATSDR's website.

Other federal guidelines limit the amount of benzene that manufacturing plants can emit, or set standards for transporting benzene on the nation's highways. Standards have also been created for people who handle benzene on a daily basis in a workplace setting. But those guidelines are for healthy adults wearing respirators—not for children, pregnant women and other vulnerable members of the general public.

"It's a mess," said Aaron Bernstein, a pediatrician and associate director of the Center for Health and the Global Environment at Harvard School of Public Health. "I know from experience that this kind of thing is a regulatory morass."

Without specific rules to help them, health authorities confronted with oil spills usually turn to these disparate guidelines and scientific studies to decide whether an evacuation is needed. They also take into account the unique characteristics of each spill, including the proximity of the oil to homes and weather conditions that can affect how quickly the fumes dissipate.

During the Michigan spill, that burden fell to Calhoun County public health director Jim Rutherford, who had more experience coordinating food inspections and school nurse programs than handling chemical disasters. His department didn't even have the proper air monitoring equipment, so he turned to state and federal regulators for help. Their instruments measured benzene readings in the nearby community that ranged from less than 50 ppb to 200 ppb. Data gathered far from homes but directly over patches of spilled oil showed benzene concentrations of more than 6,000 ppb.

But none of the regulators could give Rutherford a definitive answer on whether to evacuate, because none of the existing guidelines applied to their specific situation. Finally, Rutherford called for a voluntary evacuation based on benzene readings that spiked above 200 ppb. He lifted the recommendation about three weeks later, after benzene readings were consistently below 6 ppb.

In Arkansas, health officials decided that Mayflower residents could return to their subdivision when benzene levels in and around their homes dropped to below 50 ppb. (Most of the 22 evacuated homes have been cleared for re-entry, although none of the families have returned.) But people nearby complained of headaches, nausea and other health problems even after officials announced online that contaminants in the air were "below levels likely to cause health effects for the general population."

Dr. William Mason, chief of emergency response at the Arkansas Department of Health, told InsideClimate News that people outside the evacuation area could have left their homes if they had wanted to. In many cases, ExxonMobil—the company responsible for the spill—was willing to pay their hotel bills. Enbridge Inc., the company responsible for the Michigan spill, made a similar offer.

Robin Carbaugh, an ombudswoman appointed by Salt Lake City's mayor to help residents after the Utah spill, said voluntary evacuations can be problematic because people are reluctant to trust their own judgment. Many people complained to Carbaugh about headaches, fainting spells and even episodes of temporary blindness. But when she advised them to evacuate or seek medical attention, she said they often backtracked, saying, 'Well, I could just be making this up.'"

The problem, Carbaugh said, is that people are afraid to look weak and they don't want to make a fuss if regulators say the air is safe. Carbaugh said she sympathizes with public officials who must base their evacuation decisions on the patchwork of available science, but she also believes the experiences of individual residents should be respected and somehow incorporated into policy recommendations.

"We have a situation in Utah, Arkansas and Michigan where groups of people who don't know each other are having the same problems," Carbaugh said. "The question is, what are public officials learning from these experiences?"

Arkansas Sets Benzene Guideline of 50 ppb

After oil spills, public health decisions usually fall to county or state officials. In Mayflower, those decisions were made by the Arkansas Department of Health (ADH), which set a benzene threshold of 50 ppb.

Lori Simmons, who heads the agency's environmental epidemiology section, said the ADH calculated that a member of the general public could be exposed to air with up to 50 ppb of benzene for up to six months without long-term health effects.

InsideClimate News tried to compare that 50 ppb guideline with guidelines established by other agencies, but found that it was virtually impossible to make a direct comparison. Some guidelines were designed to protect people from certain health effects but not others. Many, like the ATSDR guidelines, come with disclaimers saying they aren't supposed to be used to define what's safe and not safe.

The EPA, for instance, estimates that people continuously exposed to 4 to 13 ppb of benzene over a lifetime have no greater than a 1 in 10,000 increased chance of developing cancer. To avoid noncancerous blood disorders, the EPA recommends that people be exposed to less than 9 ppb per day over the course of a lifetime. But the agency's website notes that the 9 ppb reference dose is "not a direct estimator of risk, but rather a reference point to gauge the potential for effects."

When it comes to worker safety, the National Institute for Occupational Safety and Health recommends a maximum average exposure of up to 100 ppb over an 8-hour workday. But because the standard applies to healthy adults who are often wearing respirators—and who are being paid for their occupational risks—health experts say members of the general public need stronger protections.

Simmons, the ADH scientist, said the actual benzene levels in Mayflower were nowhere near 50 ppb most of the time after the spill. According to data from an Exxon contractor, benzene levels outside the immediate cleanup areas have generally been below 50 ppb since April 4, although readings hit 50 ppb on April 5 and April 8. (The EPA also tested the air in Mayflower, but it only released air monitoring results for volatile organic compounds—a class of chemicals that includes benzene—not the results for benzene alone.)

The 50 ppb was set as the re-entry level for indoor and outdoor air quality and is "expected to be below any public health hazard," said ADH spokesman Ed Barham.

Experts interviewed by InsideClimate News stressed the difficulty in comparing different guidelines—but they all said ADH's benzene levels are alarmingly high.

Given that ATSDR finds 9 ppb over two weeks to be worrisome, ADH's assessment of 50 ppb over six months "doesn't make sense," said Bernstein, the Harvard public health expert. Compared to ATSDR's screening levels, the ADH allows for both a higher level of exposure and a longer exposure period, he said.

"It should definitely be a cause for concern," said Wilma Subra, an environmental consultant and a MacArthur "genius" grant recipient who regularly works with communities impacted by oil spills. "Benzene is a known carcinogen. To establish a level that high over a six-month period is unbelievable."

Brian Moench, president of Utah Physicians for Healthy Environment, a nonprofit that promotes clean air initiatives, doesn't think even ATSDR's guidelines are strong enough. Not only do they exclude cancer, but they also don't account for the fact that some people are inherently more sensitive to environmental toxins, he said. Nor do they factor in the risks faced by pregnant women and developing fetuses.

Studies show that pregnant women exposed to high levels of chemicals give birth to babies with decreased intellect and behavioral disorders, he said. "I can virtually guarantee you [the 9 ppb] has no relevance to what's safe for a pregnant mother."

Barham, the Arkansas health department spokesman, said the agency's benzene standard accounted for the vulnerability of children but not for fetal exposure.

In an email, he said ADH's levels are "site-specific dose equation[s]" that should not be compared with ATSDR's 9 ppb or 6 ppb guidelines, because those guidelines are "screening values only and are not indicators of health effects."

ATSDR's website says its guidelines "are intended to serve as a screening tool to help public health professionals decide where to look more closely...Exposure to a level above the [guideline] does not mean that adverse health effects will occur."

Nic Brescia, the EPA's federal on-scene-coordinator in Mayflower, declined to comment on Arkansas' benzene guideline. "I'm not tied into that," he said. "I'm just tied into giving ADH the [air monitoring] information, and they review it and they make the decision on if it's above or below a certain level."

Arkansas vs. Alberta Benzene Guidelines

Arkansas' benzene threshold is also considerably higher than the guidelines used in Alberta, Canada, where the heavy crude oil that spilled in Arkansas and Michigan was extracted.

Alberta has a one-hour ambient air quality objective of no more than 9 ppb. But as with many of the U.S. guidelines, that standard wasn't designed for emergencies involving the general public. Instead, it's used to model air quality from industrial sources, said Bob Myrick, air policy manager of Alberta Sustainable Resource Development, the provincial agency that manages natural resources. If a company designs a piece of equipment for a natural gas plant, for example, it would ensure that benzene levels don't exceed 9 ppb on average over the course of an hour.

Myrick said Alberta's ambient air objective was derived from existing scientific studies and established with input from industry, environmental organizations and regulators. "Based on the literature, there has been an indicator of some kind of human health effect" when benzene levels exceed 9 ppb an hour, he said.

Myrick declined to comment on whether the levels Arkansas set are too high.

Barham, the Arkansas health department spokesman, said via email that Alberta's ambient air objective is a guideline, while his agency's number is "a site-specific emergency response incident level." He did not respond when asked for further explanation.

In hopes of getting more clarification on the Arkansas levels, InsideClimate News contacted the Centers for Disease Control. Spokesman Jay Dempsey initially said the 50 ppb level Arkansas calculated is in line with ATSDR guidelines. (ATSDR is a division of the CDC.) InsideClimate then provided him with information about Alberta's standards, which he passed on to CDC toxicologists. Later, Dempsey said in an email that he could only comment on how ATSDR creates its own guidelines, and that InsideClimate would "need to discuss ADH's rationale for seemingly setting a larger concentration with a representative from ADH."

When InsideClimate News asked ATSDR about the difference between its 9 ppb screening level and the 50 ppb benzene guideline used in Arkansas, an ATSDR spokeswoman said the agency had examined Arkansas' guideline and concurred with the state's decision.

Need for a Long-Term Health Study

Some of the confusion over what's considered safe at a spill site can be attributed to the general challenge of studying toxic exposure. Krzyzanowski, the Ontario-based consultant, said the science is inherently imperfect because it is based primarily on rodent and tissue culture studies. That means rats that typically live only a few years are used to study what might happen to humans decades after a brief exposure.

"You can't test on people...and people are not rats," she said.

Scientists adjust for the uncertainty by being extra cautious, said Bernstein, the Harvard expert. ATSDR's screening levels, for example, are based on studies that found harmful health effects in rodents exposed to 10,000 ppb of benzene. The agency set a much lower concentration of 9 ppb for humans.

"If we're going to use these in some ways arbitrary tests to discern what's safe and not safe, then ATSDR is going well beyond what's considered to be safe," Bernstein said. "That being said, benzene is nasty and you don't want anyone to be exposed to it if they don't have to."

He pointed to an additional risk posed by oil spills: Most lab studies are conducted on one chemical at a time, but oil spills release hundreds of different volatile compounds. Residents in Mayflower, Marshall and Salt Lake City were exposed to all of them at once, and little is known about their combined health effects. In fact, the health effects of some of the chemicals found in crude oil haven't been studied at all.

"Science knows very little about the long-term effects of these toxic substances," he said. "How much, how often, how long is a very difficult question." The confusion over the difference between Arkansas' benzene guideline and guidelines set by federal agencies "is not new. We need to learn from our mistakes."

Both Bernstein and Moench, the Utah physician, say a full-scale follow up study should be launched in Mayflower, even though it would be costly and difficult to conduct.

"Every person who has symptoms of short-term exposure deserves to be a subject of a [long-term] study," Moench said. "Those people need to be followed for 20 years for a variety of chronic diseases—heart, brain, kidneys, you name it."

The science may be flawed, he said, but "what science we do have sends a very concerning message."

After the Utah spill, Moench was part of a physicians' group that called for Chevron, the company responsible for the accident, to fund a $2 million long-term health study. Another group demanded a $15 million long-term study.

Chevron and Salt Lake City reached a $4.5 million settlement in Sept. 2011, but the agreement did not include a health study.