According to many software developers, caching is one of the toughest topics to master in web development. The main reason for this is invalidating cache and being able to consistently keep the data you serve fresh. One can imagine how difficult this can be in a distributed architecture environment. We are not going to cover the complexity of distributed caching but rather a centralized caching in memory. We can use some in-memory data structure store like Redis, but for the sake of this exercise we will build every component ourselves and make it as simple as possible.

Without Cache

Let’s build a lightweight web application using Sinatra. We are starting with an app without Cache to point out the performance gains when making calls to the database. Let’s assume that our Database lives in a far away server and the query we are executing on it is complex and CPU intensive.

A fake database adapter would look like this:

Obviously, this is not doing any query execution. Worse, it is always returning 33. Notice that I am printing out what query is executed to see it in the logs. Let’s build the Sinatra app that would use this Database Adapter to execute a VERY expensive query:

Let’s Add Some Cache

As we can see in the previous code snippet, the VERY expensive query is what should be cached and executed as seldom as possible. The code would look like this:

The logic of the Cache is pretty straight forward and would be like the following, if the passed block has never been executed, execute it, store it as a key/value with the passed key and return the result. Otherwise, fetch the result using the passed key.

We will use a Hash to mimic a Key/Value database, we can even mimic the name :).

Let’s see what happens when we load the homepage in the browser a few times:

Perfect — we notice that the first GET request is making a Database call and all the following calls are fetching that result from the cache.

Cache With an expiration date

you might foresee the issue with our current implementation. If we get more users, the results will still be the same since it is fetched from the cache. We need to find a way to invalidate the cache in order to refresh the query’s result. We can do that by adding an expiration date to our key/value (In our case if we assume that we are not dealing with realtime requirements and it is fine if the user is not getting the updated data by the minute). So, the strategy here is to invalidate the cache if the current time is greater than the expiration date and replace the key/value by executing the block of code again.

let’s see what the logs say:

The first request is a miss. the following requests are a hit but with an expiration time this time. After 10 seconds, it is a miss again and the Cache is refreshed.

Conclusion

This is a very trivial implementation of caching that does not take into account issue we might experience when we have a distributed caching system like race conditions. The best way to understand a concept is to break it down into different parts and try to build it from scratch. This is what we have done here.

I would love to get you feedback on what would be the next step in caching and I would work on another post to implement a recommendation.