Travel Michigan contends that the spots prompted millions of trips and more than $1 billion in spending last year, citing Longwoods' research.

Yet it's impossible to independently assess the credibility of Longwoods' numbers, because neither the company nor the state will provide the public access to how it calculates returns. Siegel would not disclose the methodology the company developed to calculate the campaign's ROI, saying it's proprietary.

Nor would taxpayer-supported Travel Michigan provide direct access to how Longwoods calculated return on investment. The agency said in an email that Longwoods researchers asked visitors if they saw a Pure Michigan spot and whether it prompted their trip, and asked them to rate the state versus other states. The state said Longwoods controlled for "internal and external factors," but would not say what those were.

By Longwoods' analysis, the $12.9 million invested in Pure Michigan advertising in 2016 yielded $107.3 million in new state tax revenue — a return of $8.33 for every dollar spent.

The Mackinac Center for Public Policy, a Midland-based, free-market think tank, has been an outspoken critic of Travel Michigan, and specifically Longwoods, for its ROI findings.

The state last month parted ways with Longwoods, a decision Lorenz said was more about gaining new insights on how to direct the campaign in the future and less because of the ROI criticism. Travel Michigan plans to sign a three-year deal worth $92,500 per year, or $277,500, with Indianapolis-based Strategic Marketing and Research Insights.

The Mackinac Center contends the figures are too high to be believed, but it says it can't know by how much because the company doesn't disclose its methodology. Mackinac also claims the MEDC has a conflict of interest because it manages the program being studied and would have a reason to seek positive numbers.

"All I've seen so far is a highly secretive consultant tell the MEDC that the program is effective and has a high return on investment," said Michael LaFaive, fiscal policy director at the Mackinac Center. "Both the consultant and the MEDC have a reason to puff up the success of this program."

The Mackinac Center released a report in 2016 using its own statistical tool to calculate Pure Michigan's ROI, looking at the impact of tourist spending on hotels and accommodations, arts and entertainment and amusement and recreation. Results showed little impact on the individual industry sectors and a negative net return on investment.

Lorenz characterized the Mackinac Center's criticism as political, noting that it had opposed MEDC tax incentive programs in the past.

"People always focus on the (ROI) numbers. I never have," he said, adding that he finds the insights gleaned from the research more important, such as learning that Michigan is highly regarded for its nature and urban experiences but not so much for arts and culture.

"People also assume that we've been out looking for research that only helps us make us look good," Lorenz added. The return-on-investment calculations are "something we have to do because it's expected, and it does give us an idea of what's happening."

He said the numbers are estimates, because there is no accurate way to track every visitor, what brought them to Michigan or their spending in the state. But Lorenz said the size of Longwoods' sampling — of 4,000 travelers from the Midwest and nationally — offers a "scorecard" to measure how the brand is recognized outside of Michigan.

Ninety-eight percent "of people who look at things in a fair manner will look at what we do and say it's really making an impact," he said in response to the Mackinac Center's criticism.

Siegel said Longwoods' data are conservative on purpose.

"When you're there talking about public funds being used for advertising and marketing, it's very contentious," he said, "so we've seen more scrutiny in this area than anything else we've ever done."

The purest way to measure the impact of tourism marketing would be to run the campaign in one market and not in another to control for exposure and to see if tourist activity increases, said marketing professor David Reibstein of The Wharton School at the University of Pennsylvania.

Another way is to track tourism activity as a campaign is rolled out nationally to new markets, said Reibstein, who is not familiar with the Pure Michigan campaign. One challenge with that approach in Midwestern markets, in particular, is controlling for prospective travelers in these states who may be more likely to visit Michigan anyway.

Still, he said, organizations would be "irresponsible" if they didn't attempt to quantify ROI on public spending programs.

"They are estimates, but you need these estimates. What are you going to do in the absence of them?" Reibstein said. "Let's say we spend and the study shows we have a negative ROI. Would we want to know that? And the answer is absolutely."

The Mackinac Center argues that if lawmakers will not stop funding Pure Michigan, they should at least have an independent state agency conduct an independent audit of the campaign's performance and account for the opportunity costs, or tradeoffs, that come with diverting tax dollars to tourism promotion.

Industry partners, including visitors' bureaus across the state, should fund Pure Michigan advertising, said LaFaive of Mackinac, rather than relying on taxpayers.

"What business that knows and believes they're getting an $8.33 return for every dollar invested on their advertisement is going to pass that (return) up?" LaFaive said. "Yet the industry is saying, 'No thank you, we'll make the taxpayers continue to pay for it.' There's sort of an implicit admission that they recognize that the advertising is not that effective."

Not true, say tourism directors from Detroit and Grand Rapids.

They say regional visitors' bureaus have limited budgets that stretch farther with state dollars, because the MEDC matches their contributions toward their own locally focused commercial.

"It's helped push our leisure business, for sure," said Doug Small, president and CEO of Experience Grand Rapids, which he said contributed $900,000 toward its $1.8 million total Pure Michigan budget. "I don't have the budget to get the same impact, but if I spend $900,000, Pure Michigan matches that for me. … Why wouldn't I do it through them?"

Small said the Mackinac Center's ROI study looked at three industry sectors, while other industry research includes as many as nine to evaluate impact.