A rigorous new study of workplace wellness programs suggests that they may not be as effective as employers hope, at least in the short term. Employees assigned to receive wellness training did become more conscious of diet and exercise, but didn’t score better on various health measures or spend less on health care.

The 80 percent of large US employers that offer workplace wellness programs hope that the benefit will be a win-win situation. The programs (which usually include health assessments and info sessions) are supposed to help employees stay healthier, which would also lower their health care costs, saving money for the company. In a study published today in the Journal of the American Medical Association, researchers studied the effect of workplace wellness programs at BJ’s Wholesale Club, a warehouse retail chain. After 18 months, employees who went through the program were more likely to report that they were exercising and trying to maintain their weight, by 8.3 percentage points and 13.6 percentage points, respectively. Yet they were not significantly different from other employees when it came to other outcomes, like body mass index, cholesterol and blood pressure, or job performance and health care spending. ”There was a lot of hope that workplace wellness programs would reduce health care spending and absenteeism within a year or two, but we do not find such returns,” wrote study co-author Katherine Baicker, dean of the University of Chicago Harris School of Public Policy, in an email to The Verge.

“Expectations that wellness programs might generate a large financial return on investment in the short run might be overly optimistic.”

The study is interesting because it is one of the few that look at whether workplace wellness programs actually cause changes. Most previous studies just observe the health of employees who voluntarily join a workplace program versus those who don’t, which means that any difference in results might be because the people who voluntarily join programs are different (for example, healthier) than people who didn’t join them to begin with. In this study, the researchers randomly assigned employees at 20 BJ’s worksites to receive wellness training from a dietician, while it was “business as usual” at the other 140 sites, according to study co-author Zirui Song, a professor of health care policy and medicine at Harvard Medical School.

Song is careful to note that these results should not be generalized to all workplace wellness programs. “This is one study of one workplace wellness program with one employer,” he says, and many factors can vary. For example, the content of workplace wellness programs can differ, and the results of workers at BJ’s (who tended to be lower-income) may not be representative of higher-income employees participating in workplace wellness programs. And while one of the strengths of the program is that it’s geographically diverse and covers a lot of people, that also means there are lots of little factors that are different and might cause different outcomes.

Still, the study adds to the growing body of research suggesting that workplace wellness programs aren’t very effective. Last year, researchers published results from a randomized workplace wellness study at the University of Illinois at Urbana-Champaign. Of the 5,000 people who participated, about 3,300 were randomly assigned to be offered health screenings and wellness education, while the others were not. The study concluded that the program didn’t seem to have much effect on total medical spending, employee productivity, or health behavior in the first year.

For Song, the implications of this study may depend on what an employer hopes to achieve from the wellness program. Employers who value behavior change, for example, might be pleased that employees in the program became more attentive to diet and exercise, but others who care more about health care savings may be discouraged. The overall takeaway, says Song, is that wellness programs can change health behaviors in the near term, but “expectations that wellness programs might generate a large financial return on investment in the short run might be overly optimistic.” His team is now working on a follow-up to the study to look at the results beyond 18 months so that we can have more answers on the long-term effects of wellness programs.

Update April 16, 2019: This article has been updated to include a quote from study co-author Katherine Baicker.