Guide to Caching in Rails using Memcache 1

The post intends to cover the topics and tools that can help in implementing Memcache as caching store in Rails and debugging issues with it.

To help newbies grasp it from start, it also provides references to installing and validating the memcache install without Rails too.

“Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering”

Installing Memcache

Mac OS X Snow Leopard brew install memcached

Ubuntu sudo apt-get install memcached



More information about Installing Memcache can be found here

Starting Memcache

memcached -vv

-v option controls verbosity to STDOUT/STDERR. Multiple -v’s increase verbosity. Memcache Settings can be inspected in detail using echo "stats settings" | nc localhost 11211 . Get the thirst quenched more at http://linux.die.net/man/1/memcached or man memcached for all options.

Connecting to Memcache Server through Telnet

telnet localhost 11211 can be used to connect locally to run/inspect memcache data store

can be used to connect locally to run/inspect memcache data store Its recommended to follow Using Memcache Commands to get a full hang of memcache commands

For the sake of completeness, here are three main commands ( store/retrieve/remove key/values) Write to Memcache (Value need to be entered in new line) set abc 0 60 5 # where abc is the key, 60 is duration in seconds and 5 is length of data hello # data to store in abc key Read from Memcache get abc Delete from Memcache delete abc

key/values)

Implementing and Using in Rails

Installing Dalli (Memcache Client)

Installing Memcache Dalli Client Include gem 'dalli' in Gemfile & Run bundle install

Dalli Client Api Doc

Configure Memcache as cache store in Rails

Set perform_caching and specify cache_store in the environment file (e.g. RAILS_ROOT/config/environments/production.rb) config.action_controller.perform_caching = true config.cache_store = :dalli_store, { :namespace => “my_project”, :expires_in => 1.day, :socket_timeout => 3, :compress => true } Detailed explanation of Dalli Client options can be found here

and specify in the environment file (e.g. RAILS_ROOT/config/environments/production.rb)

Getting Ready for the Fight/Debugging

IRB as always helps us get started: require ‘rubygems’ require ‘dalli’ CACHE = Dalli::Client.new(‘127.0.0.1’, { :namespace => “my_project”, :expires_in => 3600, :socket_timeout => 3, :compress => true }) # Options are self-explanatory CACHE.set(‘key’, ‘value’, 180) # last option is expiry time in seconds CACHE.get(‘key’) 2

as always helps us get started: Memcache does not provide any command to list all keys present but at times seeing the keys and their expiry time can be a life saviour. Here is a Ruby Script to list all memcache keys Sample Output: id expires bytes cache_key 1 2014-01-22 19:05:09 +0530 5 my_project:key …

Using Memcache to store expensive calculations and avoid recalculations

We can store expensive calculations in memcache and retrieve it next time if expensive calculation expected to return the same result else recalculate and store again.

Rails provides helpers like Rails.cache.read to read cache, Rails.cache.write to write in cache and Rails.cache.fetch to return result if present in cache else write in cache and return the result Rails Cache Store Guide

to read cache, to write in cache and to return result if present in cache else write in cache and return the result Code Snippet to pass a block to evaluate if key not present in cache, else return the value of key # File: application_controller.rb # Always Calculate if caching is disabled # Calculate the result if key not present and store in Memcache # Return calculated result from Memcache if key is present def data_cache(key, time=2.minutes) return yield if caching_disabled? output = Rails.cache.fetch(key, {expires_in: time}) do yield end return output rescue # Execute the block if any error with Memcache return yield end def caching_disabled? ActionController::Base.perform_caching.blank? end

Using data_cache helper created in last step in application_controller.rb # File: posts_controller.rb respond_to :json def index # Create cache to expire after 3 minutes all_posts_for_a_category = data_cache("#{@category.name}", 3.minutes) do @category.posts.all end respond_with(posts: all_posts_for_a_category) end # Post#index - Before Filter def find_category @category = Category.where(id: params[:category_id]).first respond_with({message: 'Category Not Found'}, status: 404) if @category.blank? end

Clearing Cache

Auto-Expire after a specified time

Memcache accepts an expiration time 3 option with the key to write and only returns the stored result if key has not expired yet, else returns nothing. Rails.cache.write(key, value, {expires_in: time}) Rails.cache.fetch(key, {expires_in: time}) { # Block having expensive calculations }

option with the key to write and only returns the stored result if key has not expired yet, else returns nothing.

Faking Regex based Expiry

Unfortunately, Memcache does not allow listing of all keys so, apparently, there is no way to partially expire a subset of keys. Memcache only provides a delete method for the specified key restricting us to, beforehand, know the key to delete.

But, there is a nice trick to partially expire a subset of keys Faking Regex Based Cache keys in Rails 4 . To achieve it, we have to focus more on keyword subset and defining subset is the Nirvana to crack the problem. Introduce a dynamic component in all the keys which we want to treat as subset Have a way to control the change of dynamic component When dynamic component changes, a new key is created and stored in Memcache which would be referred for data from now. The old key still remains in memcache and has still not expired but it stores the stale data and fresh data is hold by new key which we are referring in all of our interactions. Memcache would eventually kick the old key with stale data out (uses LRU algorithm to allocate space for new data) In the snippet below, we have added dynamic component last_cache_refreshed_at in Category which can be selectively changed to expire cache of particular categories instead of all # File: posts_controller.rb def index # Update Category#last_cache_refreshed_at anytime to force the block evaulation # again and storing the new key, hence expiring the last key by effectively not # refering it for data and consulting new key for fresh data # Time converted to integer to avoid sanitization hassle all_posts_for_a_category = data_cache("#{@category.name}-#{@category.last_cache_refreshed_at.to_i}", 3.minutes) do @category.posts.all end respond_with(posts: all_posts_for_a_category) end

.

Writing Test Cases

Enable Caching with Rspec metadata # File: spec/spec_helper.rb # describe "Action", caching: true {} # Above blocks would toggle the caching behaviour # default caching behavior is OFF # Use `bundle exec rspec -t caching:true spec` to run only caching examples config.around(:each, :caching) do |example| caching = ActionController::Base.perform_caching ActionController::Base.perform_caching = example.metadata[:caching] example.run ActionController::Base.perform_caching = caching end # Using `bundle exec rspec spec` would exclude caching:true examples # to run from test suite config.filter_run_excluding :caching => true

Caching Test Example (Dependency: Memcache should be running) # File: spec/controllers/posts_controller_spec.rb # Use `bundle exec rspec -t caching:true spec` to run only caching examples context 'Caching Enabled', caching: true do before(:each) do # last_cache_refreshed_at would ideally be handled through before_create callback # by setting current time as value to it # Assumption: Current Time in seconds = 1271325600 @category = Category.new(name: 'Ruby', last_cache_refreshed_at: Time.current) @category.save.should be_true @post = @categroy.posts.build(content: 'Expiring Cache is tough !!!') @post.save.should be_true end after(:each) do @post.delete @category.delete end it "should return posts" do @category.last_cache_refreshed_at.should eq(1271325600) # Check Memcache doesnot have key Rails.cache.fetch("Ruby-1271325600").should be_nil get :index, {category_id: 1} response.should be_ok response.body.should == {success: true, posts: @all_posts}.to_json # As caching is enabled, key should be created in memcache Rails.cache.fetch("Ruby-1271325600").should_not be_nil # Invoke an action to refer a different key in memcache for same action # Assumption: Current Time in seconds = 1456782340 @category.update_attributes(last_cache_refreshed_at: Time.current).should be_true @category.last_cache_refreshed_at.should_not eq(1271325600) # Old key is still present in memcache as still not expired Rails.cache.fetch("Ruby-1271325600").should_not be_nil get :index, {category_id: 1} response.should be_ok # New key is generated even though old key has still not expired # after the specified time, old key would expire Rails.cache.fetch("Ruby-1456782340").should_not be_nil end end

Gotchas

Using touch command to update expiry cache time (Fixed in Memcache 1.4.14)

command to update expiry cache time Check Memcache Version using telnet client Execute stats Look for STAT version 1.4.13

