E-cigarette users’ pee contains more heavy metals as well as signs of exposure to carcinogens and other irritating chemicals than the pee of non-vapers, hinting that the habit might come with serious health risks. At the same time, vapers who switched completely from cigarettes still had lower levels of these substances in their urine than their smoking peers, new research says.

The study, published today in the journal JAMA Network Open, didn’t measure the risks from exposures directly — only the materials the participants peed out. But the findings may help with the ongoing debate about whether e-cigarettes are safer than cigarettes. After all, that’s a major selling point for the vape industry, which markets e-cigarettes as a less risky alternative. The problem is that they’re still largely unregulated, so we don’t know for certain what chemicals they contain or what those chemicals’ long-term health effects could be.

It’s important to find out: about 6.7 percent of adults reported using e-cigarettes in the past 30 days in a recent study, and 3.6 million high school and middle school students in the US are vaping, according to the CDC. That’s where today’s study comes in: researchers led by Maciej Goniewicz, a tobacco researcher at the Roswell Park Comprehensive Cancer Center, analyzed questionnaires and urine samples submitted by more than 5,100 people between 2013 and 2014 to try to spot patterns of chemical exposures in their pee.

“We just don’t know how unsafe they are yet.”

It’s a massive study, which is why it’s important, says Gideon St. Helen, a tobacco researcher at the University of California, San Francisco who was not involved in the research. “You don’t have many big studies like that that have looked at e-cigarette use, tobacco cigarette use, and dual use, and have biomarker data for all of these people,” he says. “It’s a very thorough work,” agrees Robert Strongin, an organic chemist at Portland State University who also did not participate in the research. But he worries that people will see the results and think e-cigarettes are safe. In fact, Strongin comes to the opposite conclusion. “It confirms that they’re really not as safe as people say. We just don’t know how unsafe they are yet.”

Today’s paper comes out of the massive, long-term Population Assessment of Tobacco and Health (PATH) study, which looks at tobacco and electronic cigarette use across the US. Most of the electronic cigarette users in the study, the researchers found, were former smokers who had quit a few years before enrolling. The people who used both, known as dual-users, smoked roughly the same amount of cigarettes as the smokers but vaped less than the vapers.

Scientists at the CDC scoured the pee samples for 50 biomarkers known to show up in tobacco users, including compounds produced when the body breaks down nicotine. Investigators looked for heavy metals and markers for carcinogens that could raise the risk of cancer. They also tested for signs of exposure to chemicals linked to heart attacks and irritated airways. The researchers compared the levels of these toxic substances in pee from cigarette smokers, electronic cigarette smokers, people who use both, and people who use neither.

“Harmful if swallowed, inhaled, or absorbed through the skin. Stench.”

Unsurprisingly, the people who didn’t smoke and didn’t vape showed the lowest levels of these toxic chemicals in their pee. People who used e-cigarettes had roughly 19 percent more lead, 23 percent more of the heavy metal cadmium, 20 percent more pyrene (a biomarker of carcinogens that are linked to heart attacks), and 66 percent more acrylonitrile in their urine. The safety information for people working with acrylonitrile says: “Toxic if inhaled; May cause respiratory irritation; May cause cancer; Suspected of damaging fertility or the unborn child.” People who vaped every day showed higher levels of these substances than people who puffed every once in a while.

Compared to cigarette smokers, the vapers fared a little better, but not on everything. Vapers had similar levels of the heavy metals — except cadmium — as the smokers. Of course, many of the vapers were former smokers, and since heavy metals stick around in the body for so long, those metals could just be leftovers from their smoking days. E-cigarette smokers’ pee also showed similar exposures to three volatile organic compounds: toluene, benzene, and carbon disulfide. Chemists take serious precautions to avoid inhaling these things in the lab. The safety information for people working with carbon disulfide, for example, says “Danger! This substance has caused adverse reproductive and fetal effects in animals....Harmful if swallowed, inhaled, or absorbed through the skin. Stench. May cause nervous system effects.”

“Real life moves much faster than these studies could really keep up.”

When people used e-cigarettes instead of smoking, they were exposed to lower levels of all the other chemicals, including nicotine. That’s surprising until you remember that the study participants submitted their pee samples between 2013 and 2014. That’s before higher-dose nicotine salts products like Juul took off. “These large studies take time,” St. Helen says. “Real life moves much faster than these studies could really keep up.”

Here’s the really surprising thing: e-cigarette users were exposed to lower levels of these substances only if they stuck to e-cigarettes exclusively. People who used both e-cigarettes and cigarettes didn’t reduce their risk at all. In fact, these dual-users showed higher levels of nicotine and many of the toxicants, including two heavy metals and some of the carcinogens. That’s worrying because more than half of e-cigarette users are dual-users, according to a CDC report from 2016. “It’s astounding in that they saw significantly higher levels of almost all the biomarkers that they measured,” St. Helen says.

“What are the toxicants in e-cigarettes that aren’t in cigarettes?”

There are a few limitations to the study. First, it only looks at toxicants that are known to be in tobacco, and it makes sense that electronic cigarettes would expose users to fewer of those things. The ingredients are different, but there could be potentially risky chemicals unique to the liquid nicotine recipe in electronic cigarettes that public health researchers don’t know to look for. One recent study reports that irritating chemicals can form when the ingredients in vape juices mix in the bottle, for example. “You’re not looking at the other side of it. What are the toxicants in e-cigarettes that aren’t in cigarettes? You’re starting from two different places,” Strongin says.

Of course, looking for toxicants unique to e-cigarettes might be easier said than done because there aren’t great markers for them, St. Helen says. For example, small amounts of formaldehyde have been spotted in e-cigarette cartridges and vapor, but there isn’t an easy way to test for formaldehyde exposure, he says. “That’s an issue, which is why they focus a lot on the toxicants in combustible cigarettes.”

“That is the biggest question.”

While we know that e-cigarette users are being exposed to some of these chemicals, we don’t necessarily know if these levels are high enough to be concerning over a lifetime of use. “That is the biggest question that, with this study, we unfortunately we don’t have the answer to right now,” Goniewicz says. Risk depends on dose: inhaling water is fatal if you inhale enough of it. And for some of those chemicals, that risk could build over time for people who start vaping at a young age and continue vaping for the rest of their lives. That’s why, going forward, Goniewicz and his team want to look for signals of the health problems, like markers of inflammation, and not just the chemicals that might contribute to them.

Still, the take-home point is that vaping is probably less risky than smoking. (“It would be almost impossible — if you tried — to make a legal product that was less safe than a cigarette,” Strongin says.) But vaping is only less risky than regular cigarettes if people switch completely to electronic cigarettes, the study says. Using both might actually increase health risks. And vaping is still certainly riskier than inhaling nothing at all. “If you don’t smoke, don’t use electronic cigarettes, because you might be exposed to toxicants,” Goniewicz says. “If you are a smoker and you consider using electronic cigarettes as a quitting method or to reduce your health risk, you have to switch completely.”