Melissa Cole (u/MiamiZ)

Senior Software Engineer

Earlier this year, we unveiled our new user profile pages, but because the new pages launched as an opt-in beta feature, we had to keep our old profile pages intact as well. In this post, we’re going to give you a closer look at how we loaded both the new and old profile pages on different stacks.

The motivation

Part of my team’s work involved creating profile pages using Reddit’s new frontend stack (React + Redux). As part of a product decision, we decided to allow users to opt into the new profile experience during the beta period. The old profile pages remain on Reddit’s app servers, but the new profile pages are served by a completely different stack and service. Because of this, we needed a way to programmatically redirect to the appropriate stack depending on the profile being visited and whether it is part of the beta or not. We needed to do this higher up in the stack than either type of app server, which means we had to handle this logic at Fastly, our content delivery network (CDN). Since we were the first ones to use this new stack for a page in production, we were also the first ones to tackle redirecting traffic for a specific page to the new stack via Fastly. Fastly uses a derivative of the Varnish Configuration Language (VCL) to handle and modify requests, which allows us tremendous configuration flexibility.

The development hack

When we were still developing, we were able to test the new stack by first visiting a secret URL that would attach a cookie to our browser session. This special cookie then redirected any subsequent requests to a profile that we had added to Fastly’s whitelisted URLs to use the new stack’s servers. We were then able to load our test accounts with the new stack. For the alpha release, we just removed the secret URL requirement so that everyone was redirected. We were able to sustain this method since we only released profiles to a few select users and could hardcode a handful of URLs easily.

Sample VCL snippet for the temporary redirect

sub vcl_recv { set var.is_whitelisted_profile_page = ( req.url.path ~ "(?i)^/user/Montanita/?$" || req.url.path ~ "(?i)^/user/totallyvalidtestaccount1/?$" || req.url.path ~ "(?i)^/user/totallyvalidtestaccount2/?$" || req.url.path ~ "(?i)^/user/totallyvalidtestaccount3/?$" ) if (var.is_whitelisted_profile_page) { set req.backend = new_reddit_frontend; } else { set req.backend = reddit; } }

However, when we wanted to release profile pages into an opt-in beta, this wasn’t sustainable as we couldn’t hardcode thousands of profile URLs. We also wanted users to have an immediate opt-in flow rather than submitting a request, so this would have been an issue for even tens or hundreds of profiles. Since we didn’t want to route all profiles over to the new stack, we needed another method.

The solution

Rather than rely on hardcoded URLs to determine the backend in Fastly, we moved the selection process to the main Reddit servers with an extra API call. To do this, we utilize the ability in the VCL code to restart requests in flight while retaining variables on the request object, such as storing the original URL and a special backend header that specifies which backend to use.

Once we’ve determined that the incoming request matches the profile page URL regex and doesn’t have the special backend header attached, we create the special backend header on the request object initialized with a sentinel value. We also attach the original URL to a header on the request object since we’re about to change the request URL. The request URL is rewritten to swap out the “/user/” portion of the URL with the address of an API endpoint while still keeping the username portion so that the backend can be determined.

When the request goes through to the API endpoint, the username is fetched from the URL and the Reddit app checks to see if the profile meets all of the conditions to be loaded in the new stack (such as having opted into the new profiles, is not deleted or banned, and matches the URLs that we support in the new frontend stack) to determine which backend to use. A special backend header is written onto the response object with the chosen backend, falling back to the original Reddit backend.

When Fastly receives the response, the special backend header from the endpoint is copied to the request object, the request object’s URL is rewritten to be the original URL again, and the request is restarted. At this point, the special backend header on the request specifies which backend to use, so we just set the backend and let the request continue through on the chosen backend.

Sample VCL snippet to fetch the backend location

sub vcl_recv { # This is the first time the request for a profile page # has come through (no special backend header attached) if ( !req.http.dynamic-backend && req.url.path ~ "(?i)^/user/(.*)/?(.*)" ) { set req.http.dynamic-backend = "sentinel"; set req.http.original-url = req.url; set req.url = regsub(req.url, "/user/", backend_endpoint_url); set req.backend = reddit; } # The backend has already been determined, so use it if (req.http.dynamic-backend != "sentinel") { set req.backend = req.http.dynamic-backend; unset req.http.dynamic-backend; unset req.http.original-url; } } sub vcl_deliver { # If we have the special header in the response object, # put that in the request object and restart the request if (resp.http.dynamic-backend) { set req.url = req.http.original-url; set req.http.dynamic-backend = resp.http.dynamic-backend; restart; } }

The implementation adds a little bit of overhead since Fastly now makes 2 requests per profile page request (one to fetch what server to use and one more to actually load the profile page from the correct app). The p99 overhead is around 12 ms (so 99% of requests are quicker than this) and the p90 overhead is 7.5 ms. The latency added was pretty small in most cases, and we can still improve the response times by adding caching in Fastly.

Another method we could have tried is implementing a new service that returns which backend to use. The benefit of this is that it could have been reusable by other pages and services. However, it would have taken longer to implement, required storing which users opt in to the new profile, and in some cases it still would have made another request to the main backend to get more data to make the choice, so the downsides outweighed the benefits.

Conclusion

Using custom VCL in Fastly is powerful and allows us the flexibility of routing to different stacks, which is useful since we’re starting to migrate our frontend to an entirely new stack. With this change, we have been able to dynamically route 75,000 profiles of users that have opted into the new profile page beta. This implementation has already come in handy in employee testing the redesign (coming soon to a desktop site near you). In the meantime, if you haven’t made the switch to the new profile page on your Reddit account, you can follow this guide here!

Many thanks to u/rram for all of the VCL help!