.11

February 2015

Launch Of Our Synchronized Worldwide Infrastructure

During this month, we accomplished the vision we had been working towards since April 2014, “worldwide expansion to better serve our users”.

This network was composed of 12 different locations: US-East (Virginia), US-West (California), Australia, Brazil, Canada, France, Germany, Hong Kong, India, Japan, Russia and Singapore. On top of all that, we also launched our “Distributed Search” feature. With this feature, in a few clicks, you are able to set the locations in our network where your data should be automatically duplicated. You get to use the same API as before and queries are automatically routed from the end-user’s browser or mobile application to the closest location.

This was a huge step in reducing latency for end users and improving our high availability via worldwide distribution of searches. Serving international users from one location leads to a very different quality of service because of distance and saturation of Internet links. We now provide our users a way to solve that! On top of reducing latency, this improved the high availability of our search infrastructure because it was no longer dependent on a single location. It’s worldwide!

Some people compare our DSN (Distributed Search Network) to a CDN (Content Delivery Network), but it is very different. We don’t store a cache of your frequent queries on each edge. We store a complete replica of all your data. We can respond to any query from the edge location itself, not just the most popular. This means when you select three POPs (US-East, Germany, Singapore), users in Europe will go to the Germany location, users in Asia will go to Singapore, and users in America will go to US-East. All POPs will respond to queries without having to communicate to another edge. This makes a huge difference in terms of user experience and high availability.

In order to support this change, we updated our retry logic in our API clients to target theAPPID-dsn.algolia.net hostname first, which is routed to the closest location using DNS based on geoip. If the closest host is down, the DNS record is updated to remove the host in less than one minute in order to return the next closest host. This is why we use a low TTL of 1 minute on each record. In case of failure, if the host is down and DNS has not been updated, we redirect traffic to the master region via a retry on APPID-1.algolia.net,APPID-2.algolia.net and APPID-3.algolia.netin our official API clients. This approach is the best balance we have found between high performance and high availability, we only have a degradation of performance during one minute in case of failure, but the API remains up & running!