By offering the taste of sweetness without any calories, artificial sweeteners seem like they could be one answer to effective weight loss. The average 12-ounce can of sugar-sweetened soda delivers about 150 calories, almost all of them from sugar. The same amount of diet soda—zero calories. The choice seems like a no-brainer.

The American Heart Association (AHA) and American Diabetes Association (ADA) have given a cautious nod to the use of artificial sweeteners in place of sugar to combat obesity, metabolic syndrome, and diabetes, all risk factors for heart disease.

While they are not magic bullets, smart use of non-nutritive sweeteners could help you reduce added sugars in your diet, therefore lowering the number of calories you eat. Reducing calories could help you attain and maintain a healthy body weight, and thereby lower your risk of heart disease and diabetes.,

As with everything, there’s more to the artificial sweetener story than their effect on weight. To learn more about them, I spoke with Dr. David Ludwig, an obesity and weight-loss specialist at Harvard-affiliated Boston Children’s Hospital. He has a keen interest in products designed to help people lose weight at keep it off. And what he has learned about artificial sweeteners worries him.

All artificial sweeteners are not created equal

The FDA has approved five artificial sweeteners: saccharin, acesulfame, aspartame, neotame, and sucralose. It has also approved one natural low-calorie sweetener, stevia. How the human body and brain respond to these sweeteners is very complex.

One concern is that people who use artificial sweeteners may replace the lost calories through other sources, possibly offsetting weight loss or health benefits, says Dr. Ludwig. This can happen because we like to fool ourselves: “I’m drinking diet soda, so it’s okay to have cake.” The AHA and ADA also added this caveat to their recommendation.

It’s also possible that these products change the way we taste food. “Non-nutritive sweeteners are far more potent than table sugar and high-fructose corn syrup. A miniscule amount produces a sweet taste comparable to that of sugar, without comparable calories. Overstimulation of sugar receptors from frequent use of these hyper-intense sweeteners may limit tolerance for more complex tastes,” explains Dr. Ludwig. That means people who routinely use artificial sweeteners may start to find less intensely sweet foods, such as fruit, less appealing and unsweet foods, such as vegetables, downright unpalatable.

In other words, use of artificial sweeteners can make you shun healthy, filling, and highly nutritious foods while consuming more artificially flavored foods with less nutritional value.

Artificial sweeteners may play another trick, too. Research suggests that they may prevent us from associating sweetness with caloric intake. As a result, we may crave more sweets, tend to choose sweet food over nutritious food, and gain weight. Participants in the San Antonio Heart Study who drank more than 21 diet drinks per week were twice as likely to become overweight or obese as people who didn’t drink diet soda.

But you say you can give up diet drinks whenever you want? Don’t be so sure. Animal studies suggest that artificial sweeteners may be addictive. In studies of rats who were exposed to cocaine, then given a choice between intravenous cocaine or oral saccharine, most chose saccharin.

What’s your definition of safe?

Whether non-nutritive sweeteners are safe depends on your definition of safe. Studies leading to FDA approval have ruled out cancer risk, for the most part. However, those studies were done using far smaller amounts of diet soda than the 24 ounces a day consumed by many people who drink diet soda. We really don’t know what effect large amounts of these chemicals will have over many years.

And there are other health concerns beside cancer. In the Multiethnic Study of Atherosclerosis, daily consumption of diet drinks was associated with a 36% greater risk for metabolic syndrome and a 67% increased risk for type 2 diabetes. Aren’t these diseases that artificial sweeteners may help prevent in the first place?

Back to sugar?

Maybe sugar isn’t too bad after all. It’s all in how it’s packaged.

“Sugar-containing foods in their natural form, whole fruit, for example, tend to be highly nutritious—nutrient-dense, high in fiber, and low in glycemic load. On the other hand, refined, concentrated sugar consumed in large amounts rapidly increases blood glucose and insulin levels, increases triglycerides, inflammatory mediators and oxygen radicals, and with them, the risk for diabetes, cardiovascular disease and other chronic illnesses,” Dr. Ludwig explains.

I think I’ll have a glass of water and an apple.