Aspartame, saccharin, sucralose—sugar alternatives go by many names, but share an almost irresistible promise: all the sweetness of sugar without the calories, weight gain and increased risk of diabetes that comes with uncontrolled amounts of sugar in the blood.

But studies on artificial sweeteners and weight loss—as well as research about whether sugar substitutes helped people avoid metabolic disorders like diabetes—have been mixed. And in a paper published Wednesday in Nature, Dr. Eran Elinav from the Weitzmann Institute of Science in Israel found that the sugar stand-ins actually contribute to changes in the way the body breaks down glucose. How? Fake sugars aren’t digested and therefore pass directly to the intestines, impacting the millions of invisible bacteria that live in our gut. And when he and his colleagues gave seven people who didn’t normally use artificial sweeteners the sugar substitutes for seven days, about half of the people showed higher blood glucose levels after just four days.

MORE: 5 Steps to Quitting Artificial Sweeteners

“What our comprehensive genetic profiling of the microbiome pointed to is that exposure to artificial sweeteners directly impacts the microbes,” Elinav says. “We found that the artificial sweeteners we think of as beneficial and that we use as treatment or preventive measures against obesity and its complications are contributing to the same epidemics they are aimed to prevent.”

In the intestines, gut microbes are hard at work, pulling out some nutrients from food that are helpful in stopping tumor growth, for example, and squirreling away others to store as energy for later use. But while artificial sweeteners aren’t absorbed by our own cells, they may be absorbed by our bacteria—and when that happens, things appear to go haywire.

Higher amounts of the sweetener substitutes, Elinav and his team found, can change the makeup of these bacterial communities. And that in turn can change how those bugs behave, leading to weight gain and poorer glucose breakdown. These alterations in intestinal bacteria were the same as those in a group of 400 people who reported using artificial sweeteners—and those changes were the same in mice as well.

MORE: Why Your Brain Isn’t Fooled By Sugar Stand-Ins

In the mouse studies, Elinav’s team found that the artificial sweeteners pushed one particular group of bacteria, Bacteroides, to thrive, while inhibiting growth of another, Clostridiales. Bacteroides are the microbial equivalent of hoarders, hungrily pulling energy out of food and squirreling it away as fat. The end result of a Bacteroides-heavy gut is a physically heavy gut as well. In studies by other research groups, its dominance, and the resulting drop in diversity of other microbes, is typical of obese people compared to normal weight individuals.

MORE: 7 Not-So-Sweet Lessons About Sugar

The metabolic consequences were also dramatic in both the mice and people studied. In the mouse experiments, animals who were fed the same dose of saccharin that the U.S. Food and Drug Administration considers safe for daily use showed a drop in their ability to break down glucose. When he gave those mice antibiotics, their ability to break down glucose returned to normal, suggesting that wiping out the abnormal balance of bacteria could return the animals back to a healthier state.

And to confirm that the changing microbial communities were indeed responsible for the glucose changes, he also transplanted fecal samples from the people using artificial sweeteners into mice whose own guts had been wiped clean. These mice then developed the same abnormalities in glucose breakdown that the human donors and the mice who were fed saccharin did—even though they never actually ate artificial sweeteners. Simply harboring the microbes that had been exposed to the sweeteners was enough to disturb their glucose metabolism.

MORE: Can Sugar Substitutes Make You Fat?

The good news is that as easily as the gut microbiome can shift toward an unhealthy state, it can just as easily be brought back into line with the proper balance of bacterial communities. The best way to do that isn’t clear yet, but, says Eran Segal, a co-author of the study and a professor of computer science and applied mathematics at the Weitzmann Insttitute, “We believe that the situation today at the very least needs to be re-examined. We were able to induce glucose intolerance in a few days in some individuals, so this massive, unsupervised and unregulated use [of artificial sweeteners] should at the very least be reassessed and perhaps re-examined in additional studies.”

Elinav, for one, isn’t waiting. Based on his findings, he’s stopped adding artificial sweeteners to his coffee.

Contact us at letters@time.com.