If the SEO experiment is successful, then there will be a sustained increased level of traffic in the enabled group. The formula we use to calculate the additional gain in traffic from shipping this experiment is:

Traffic gain = ((avg enabled after - avg enabled before) - (avg control after - avg control before)) / Experiment group size

For this experiment:

((68.7K - 45.8K) - (50.7K - 49.4K)) / .1 = 215K additional traffic per day when fully shipped

From this basic SEO experiment framework, which is essentially just a bucketing and logging function, you can upgrade it now or later by automating the data aggregation and graph displays. Another extension is that since you have a mapping of experiment group to user ids, you can also compare downstream events such as signups or logins. Finally, another important thing to note is that we require that all experiments that change the layout or content of unauth pages to be run in conjunction with a SEO experiment, in order to avoid inadvertently dropping traffic. This is key to preserve your hard earned traffic. In order to apply this at scale, you should build automated SEO experiments into your unauth experiment framework.

High leverage SEO success factors

There are many success factors that affect Google search rankings, and it changes all the time as Google’s algorithms change. However, some of these factors are hard to directly control. Ranked roughly by ROI (low time cost, high impact), they are titles/descriptions, additional relevant content, internationalization, mobile (AMP), and performance. I’ll talk a bit more about titles and descriptions and content since they are the easiest to optimize and you should try them first. Since each factor probably has enough content for its own blog post, I’ll leave the others for a later time.

Titles and descriptions

Titles and descriptions are the easiest way to increase your SEO traffic because as long as you have an SEO experiment framework and you can run a lot of experiments easily, each variant just requires a new copy. The impact of these experiments can be quite large too, as I’ve seen small copy tweaks lead to over 10% increases in traffic. You should never assume that your current title is the best possible title because you’ll usually be surprised if you run a few more variants. Also as you run these experiments, you learn more about the type of titles that work well for your product.

When coming up with new titles to test, you have to consider two main audiences: users and googlebot. If your title entices users and increases click through rate, Google in general will increase your traffic. If your describes the content on the page well, googlebot knows better what to rank you for and in general will increase your traffic.