What if they're all ... working together? Rob! / Flickr, CC Online advertisers are dreading the arrival of Apple's new iOS operating system for iPhone and iPad because the new version of the Safari browser will contain an ad-blocking feature. As 25% of web browsers use Safari, publishers and advertisers are worried that they might see 25% of their audience (and revenue) disappear.

Advertisers and publishers are already struggling with the increasing popularity of ad blocking software like Adblock Plus, because using an ad-blocker on your web browser makes surfing the web so much faster.

And Business Insider recently wrote about the debate over whether ads slow down web pages. Naturally, advertisers blame publishers for being unsophisticated; and publishers blame advertisers for clogging up their pages with heavy downloads.

A Business Insider reader claiming to be a former ad tech executive at a really huge, well-known news publisher then sent me a fascinating email, in which he claimed that ad tech companies deliberately serve ads slowly because everyone makes more money that way.

"Ad networks get away with a lot"

Basically, his theory is, when a reader clicks to read a story, the page calls for bids from advertisers on the ad space available. This bidding is supposed to take place in a few milliseconds. But, my correspondent says, ad tech companies hold open the bids much, much longer, so more bids come in, driving up the price. Publishers hate this because it makes pages load really slowly, giving readers a terrible experience. But it's hard to stop because everyone — publisher included — is taking a cut of the winning bid. So publishers and ad tech companies actually have an incentive to make pages load slowly.

This, naturally, sounded like an amazing story. So we checked it out.

Generally, other sources in the ad tech business tell us that this is rubbish. But a couple also admitted that there are some shady practices out there, and it is possible for this happen. "In theory he is basically correct. Publishers and the ad networks they work with have pretty effectively gamed each other. Lots of crazy s--- happens," one source told us. "It's a wild world out there and publishers are not generally very technically competent so ad networks get away with a lot."

So, before we go any further, let's be clear: There is no concrete, on-the-record evidence that the major ad tech companies hold up web pages to make more money. None of our sources — beyond the one exec who sent me an email — said they knew of this happening.

But this conspiracy theory is worth discussing because, apparently, some people in the industry believe it (even if it's not true.) And two of our sources indicated that the system could be faster but can be gamed by larger, slower players. Naturally, because Business Insider couldn't confirm the theory, we decided not to name any of the companies involved in the allegations. But they are household names in the business. We're publishing this information anyway because it illustrates the paranoia and gossip that is rife within the industry, especially as publishers are dependent for revenue on companies they believe slow down their products.

Here is the theory, from the email. Redactions are in [square] brackets:

I spent 5 years (up until 2013) as Principal Engineer and Solution Architect for [a major global news company], leading all of the web-facing teams (desktop and mobile). Almost 14 billion serves a month.

Having spent countless (countless!) hours dealing with [well-known ad tech company's] crap and their utter incompetence driving me to almost-insanity, I can confirm everything in your article. Our page load times were consistently proven to be horrific as a direct result of [this company] and every single time they refused to believe it. We even had a single "optimisation" person whose job it was to basically shout at people who were responsible for making pages slow on a daily basis - i.e. [this company] pretty much 100% of the time.

Part of the problem is that behind the scenes, [the company] actually runs a real-time automated auction for ad slots in which ad buyers (or their software really) bid for the slot. The best bid wins and the slot is placed. Naturally, there is a timeout (and not a massively conservative one) on this auction, so slow-responding bidders hold everything up, and therefore hold up the ad slot, and therefore the page. All this happens as soon as you request an ad and is designed to keep ad prices up.

My entire team of devs and testers mostly used Adblock when developing sites, just because it was so painful otherwise!

I could not persuade my source to provide proof of this allegation. (He'd never work again if he did.) But his LinkedIn profile confirmed his biographical details.

We ran this by a handful of senior, experienced ad tech execs to see if they had heard of such a thing. Two sources responded that bidding occurs within an industry-wide set timeframe of about 100 to 120 milliseconds. Late bidders just aren't allowed in otherwise. There is no need for extra time because bids can be made instantly — so no one has an interesting in extending bid times. "The big exchanges mandate a certain response time (I believe it is 100 milliseconds) otherwise buyers can’t make a bid," one source said, pouring cold water on the theory.

"This happens. For sure."

But a third source said this:

This happens. For sure.

If someone looks at an absolutely vanilla configuration, they might be inclined to say that type of thing is rubbish. The reality is that the programmatic stacks have become so cannibalistic, library hacked and side-scripted that there are sites you can go to that appear to never stop loading.

Without going too far into the woods, what you would typically expect to see as prescribed would be a set of ad calls to the publisher ad server, which controls a programmatic waterfall.

There is a way of gaming that system, using a set of 3rd party scripts on the page that interact with the ad server waterfall decisioning.

A fourth source said it wasn't technically possible:

I don’t think this claim can be correct.

Firstly, the source assumes there are many bidders that cannot respond quickly enough, and therefore there’s an incentive to slow the decision process. I don’t think this assumption is correct.

Secondly, I'm not sure gaming with the programmatic waterfall the way he describes is technically possible.

"Ad networks certainly f--- up publishers' pages all the time."

At this point, it became obvious that without specifics — and without the ability to get all my sources into the same room where they could hash this out together — it would be difficult to kill or confirm the rumour. But overall, our sources said they did not believe that advertisers and their ad tech providers had any interest in slowing down the web, because there is more money in serving ads super-fast than there is in serving them in a way that irritates users. One source added:

Clearly this guy is passionate about this. He either really knows what he is talking about or he knows very little and he is making a bunch of s--- up about these specifics. He is confusing a number of things which leads me to think the latter but who knows.

What he is talking about is what happens on a publisher's page which is a different ball of wax from ... the actual exchange bidding.

In theory he is basically correct, publishers and the ad networks they work with have pretty effectively gamed each other. Lot of crazy s--- happens. It's a wild world out there and publishers are not generally very technically competent so ad networks get away with a lot.

Short story, ad networks certainly f--- up publishers' pages all the time.