Last week, a report commissioned by the City of Austin was released which looked at the effect of the plastic bag ban in the city.

The report says that, in the two years since the Austin City Council banned single-use plastic bags, Austin reduced its annual consumption of plastic bags by nearly 75 percent. But some researchers say that’s not entirely true.

Lucy Atkinson is a researcher and an assistant professor at the University of Texas at Austin who studies politics and consumer behavior. She’s also a supporter of Austin’s plastic bag ban, so she says it pains her to say that the results of the study do not technically represent a reduction.

“We can’t say it was necessarily a 75 percent reduction,” she admits. “We can’t say without having some initial pre-bag ban measurement whether that’s actually a reduction in bag use.”

Before the bag ban was put in place, the city did not collect data about how many plastic bags were in recycling streams or littered. If you don’t know the number of bags before, Atkinson says there’s no way to know if there are fewer bags now — or if the bag ban affected the number.

“Because we don’t have that ‘Time 1’ measurement to compare it to ‘Time 2’ after the ban,” she says. “We can’t phrase it as a reduction. We just don’t know.”

So where did that "75 percent reduction” figure come from?

The study’s author, Aaron Waters, compared the number of plastic bags found during a volunteer cleanup in Austin and a volunteer cleanup in Fort Worth – a city with a similar population that doesn’t have a bag ban.

6,857 volunteers collected 8,757 plastic bags in Fort Worth's cleanup, while more than 3,000 Austin volunteers collected 1,131 plastic bags. Using the total weight of all the collected litter, the weight of the collected plastic bags and the total number of bags, Waters calculated each city's litter rate: Fort Worth's litter rate is .12%, while Austin's was .03%.

Then, Waters took each city's population from the 2010 Census and the average number of plastic bags used per year by the average American to find the equivalent number of plastic bags used per person in the city. That assumes that the single citywide cleanup was a representation of plastic bag use for everyone in the city.

Based on that calculation, the study estimates approximately 200 million fewer bags were used in Austin than in Fort Worth per year.

The study says, "since the implementation of the single-use bag ordinance, and all other considerations being the same, the city of Austin has reduced their yearly single-use plastic bag consumption by more than 197 million bags per year."

Atkinson says that isn't accurate. This study just shows the difference in plastic bags collected between the two cities.

“It’s a difference in plastic bag consumption at this one point in time between Austin and Fort Worth,” she says. “It’s a big difference, and that’s great. But what we can’t say is that this number before the bag ban, it would’ve been 200 million more bags in the city.”

Vinnie Cicchirillo, another professor at UT, had similar concerns.

“They’re using cross-sectional data, they’re comparing themselves to Fort Worth and making longitudinal inferences, but there’s no measurements beforehand," Cicchirillo says. Cicchrillo says without data from multiple years, you can't say for sure that the bag ban reduced plastic bag consumption. You can only say there are fewer plastic bags in Austin's litter stream than in Fort Worth's at this one period in time.

Plus, out all of the data used to calculate this rate, only one number is actual collected data — the population. Every other number is an extrapolation.

The study’s author, Aaron Waters, says because the city did not have data on plastic bag use in Austin before the ordinance, most of what he had to go on was anecdotal.

"I surveyed a ton of people, and everybody said based on their experience in litter abatement prior to the implementation of the ordinance, there was a serious reduction compared to now," Waters says. "People in the creeks, pulling bags out of trees, pulling bags out of waterways. There was just a lot of anecdotal evidence. That's really the main source I had to go off of that."

But Waters says as a scientist, "it’s hard for me to take stock in that evaluation,” he says. “I prefer to have data.”

Another part of the study compared the number of plastic bags in Austin recycling streams versus recycling streams of nearby towns. Waters found there were fewer bags in Austin’s recycling stream. To him, that indicates the bag ban was having an effect.