Google AMP was supposed to help publishers deal with slow-loading mobile web pages and boost consumption of articles on phones when it rolled out almost three years ago.

But publishers such as The Daily Beast aren’t seeing that promised increase in traffic on AMP to justify the lagging monetization they experience on AMP. The Daily Beast ran an A/B test where it randomly published articles in AMP and non-AMP formats. AMP articles didn’t get more page views from Google, according to analysis by lead data scientist Brad Doll.

It turns out The Daily Beast isn’t alone in having this issue. Chartbeat analyzed 159 AMP publishers using the format to see if it increased their traffic.

Only one-third of the 159 publishers Chartbeat analyzed saw a traffic boost from using AMP. But traffic among publishers using AMP rose 22% overall, hiding the fact that the majority of publishers didn’t see benefits.

Even among publishers who saw changes due to AMP, results varied widely. Individual publishers saw decreases in traffic up to 58% and increases in traffic up to 151%.

Could some publishers be following best practices others weren’t? Chartbeat looked for multiple factors that might be shared among the publishers who saw a boost from Google AMP and came up with nothing, according to its data scientist director, Christopher Breaux.

Google couldn’t say if those publishers had anything in common. But configurations of AMP do vary. AMP product manager Vamsee Jasti said a few issues with The Daily Beast’s AMP setup could improve its results using the platform it planned to walk the publisher through next week.

However, there is a reason for publishers like The Daily Beast to expect a traffic boost. Google’s internal studies show that users are more likely to click on an AMP article tagged with its lightning bolt icon, which could help traffic. Although AMP itself is not a ranking signal in Google search, Google prioritizes AMP articles within its news carousel and takes into account how fast pages load.

Un-AMP?

The Daily Beast, which adopted AMP in September for its article pages, is evaluating whether it will continue to use AMP, which hinges on three factors, according to Chief Product Officer Lauren Bertolini.

First, AMP is the only option to present publisher-controlled articles in the Google News app, which launched in May.

Second, Google said it’s making changes this summer to penalize publishers with slow mobile sites. Although its mobile web and AMP pages load at the same speed, “we need to maintain that speed as we continue to develop our product,” Bertolini said.

Third, The Daily Beast continues to work with its revenue partners to optimize AMP pages in hopes that monetization will catch up.

AMP ads

When AMP launched, it didn’t even have ads. In early versions, content loaded much faster than ads, leading to situations where a reader could scroll past an ad before it loaded.

Since then, AMP has progressed immensely, according to Jasti. “AMP has all the features to drive at parity and better,” he said.

But ad parity is not The Daily Beast’s experience. “The tighter restrictions AMP has in place remove the ability for publishers to run a comparable header bidding setup, which reduces the competitive pressure for inventory on those pages,” Bertolini said. “A handful of our premium mobile partners do not currently run with AMP, but actively fill inventory on our mobile web pages.”

Google’s Jasti recommends publishers up their ad load to achieve parity with their mobile web pages. In addition to the 100 ad network partners natively build into AMP, they can add third-party ad partners and header bidding to up revenue. A year ago, Google rolled out Fast Fetch, which makes ad requests earlier in order to render ads faster.

But the separate setup means publishers (and tech platforms within AMP) have yet another format and connection to maintain, which itself could affect how well the format monetizes.

The Daily Beast’s experience with AMP furthers the idea that publishers need to take an individualized approach to platforms.

“We made a decision not to go with [Facebook] Instant Articles and felt like AMP was a better fit,” Bertolini said. “Publishers have to be cautious about any new opportunity in the market. The question is not ‘Does this work?’ but “Does this work for us?”