Index Exchange recently confirmed they engaged in a practice called bid caching that many in the industry consider to be manipulative to buyers. They also recently confirmed they are discontinuing this practice. Our research indicates this practice was unique to Index and may have accounted for up to 50% of their filled impressions on some websites.

What Is Bid Caching?

There are two terms that are easily conflated but critically different: ad caching and bid caching.

Ad caching is an IAB-endorsed practice in which an auction is conducted in advance of an ad opportunity in order to reduce ad serving latency and improve the user experience. As an example, a mobile game might conduct an auction during gameplay in order to fill an interstitial ad that renders between game levels. Ad caching is especially useful in environments with poor internet connection speeds and unreliable connection quality.

Bid caching is the practice of utilizing a lost bid from one auction to fill a subsequent auction for the same user on the same piece of content. For example, an advertiser’s bid for a homepage ad impression might be used to fill an impression on an article page later in that user’s session. In cases where an auction captures either low bids or no bids from DSPs, bid caching enables the exchange to deliver elevated clearing prices to their publisher partners. The yield benefits of bid caching are particularly useful to exchanges in header bidding environments where multiple exchanges compete to produce the highest price for each available impression.

Ad caching conducts an auction in advance of the impression’s availability with the goal of improving the user experience. Bid caching transfers bids from one auction to another with the goal of boosting publisher yield.

How Common Is Bid Caching?

In early August 2018, we conducted a series of bidding tests through a major DSP with the goal of identifying the incidence of bid caching. Our tests delivered over 1.7 million impressions across a mix of comScore 1,000 websites that each sell their inventory through multiple header-enabled RTB ad exchanges. We served a single 300x250 banner ad to a US-based audience.

Bid caching is most easily detectable for impressions that violate a campaign’s frequency cap. In these cases, an impression serves for an auction in which the DSP should not have participated. Some frequency cap violations are unavoidable and do not necessarily indicate bid caching, but other frequency cap violations carry the hallmarks of a cached bid. To maximize the number of observed frequency cap violations, we configured our test campaign with a 1-per-lifetime user frequency cap. We then audited all cases in which a user was served more than one impression to determine if the frequency cap violation was caused by unavoidable factors or by bid caching.

Bid caching red flags

Using bidding logs and ad serving logs from our test campaign, we audited all 1.7 million impressions to detect cached bids. While it is not possible to know the inner workings of each ad exchange, we monitored for three red flags that we believe suggest the presence of bid caching:

Bid Request Page Mismatch: When an impression violates a frequency cap, is it served on a URL that is different from the URL declared in the bid request?

Sequential Page: When an impression violates a frequency cap, is it served on a URL that is different from the URL where we delivered this user's prior impression?

Time Lag: When an impression violates a frequency cap, is it served more than 10 seconds after the user's prior impression?

The presence of any one of these red flags does not necessarily indicate bid caching and might be explained by a variety of nuanced factors relating to DSP frequency cap enforcement latency, ad exchange win notification delays, and publisher data leakage protections. But when we observe an impression with two or three red flags, we have increasing confidence that the exchange is engaging in bid caching.

Bid caching examples

Based on the three bid caching red flags, we identified and manually inspected suspicious impressions. For each suspicious impression, we also inspected the user’s prior ad exposure. Here is an example of an impression that we believe is the result of a cached bid on a major US publication: