Fast Autocomplete Search Terms - Rails

___

Learn how to achieve a lightening-fast global autocompletion to a rails app and improve performance using soulmate, soulmate.js and redis

Introduction

In many cases you find that you need to introduce a global search in your rails application i.e. search multiple models at a go using one form. A good example is the autocomplete search at SeatGeek. See how nicely they autocomplete performers, events and venues in a single form, interesting isn't it? In this tutorial we will try and achieve something similar to seatgeek by building a global autocomplete search across multiple models in our rails application. To do this we will use Soulmate Gem which is actually developed by the guys at seatgeek, soulmate.js which is a jQuery front-end for the soulmate auto-suggestion gem and Redis that the soulmate gem uses as a backend.

One huge plus I think soulmate has is being able to completely separate the auto-suggestion engine from the main app. This allows it to get overwhelmed with a huge number of requests without significantly affecting the user’s experience. Take a sneak peek of the Demo App we will be creating.

Installing Redis

I'm guessing you already have Redis installed in your system by now but if not, installing it is actually very easy

On a mac, with homebrew just run: brew install redis

On a Unix/Linux base systems, for debian based systems e.g. ubuntu just run sudo apt-get install redis and on redhat based systems e.g. Fedora/Opensuse run sudo yum install redis

You can then verify if redis was successfully installed by starting redis server. On you terminal run:

$ redis-server

You should have something similar to

Getting started

Now that we have redis up and ready, lets start up by creating a rails 4 application to try out soulmate on.

$ rails new fast-autocomplete

Now lets add soulmate gem to our gemfile. I'm also adding the faker gem to help us populate our database with test records instead of creating them and then run bundle install to install our Gems.

Loading Gist

Creating models and adding test data

Now that we have our Gems installed, lets quicky scaffold two models. We will have a noun and a verb models to test this. We will easily populate this tables using the Faker gem shortly. On your terminal

$ rails g scaffold noun name:string $ rails g scaffold verb name:string

Next run rake db:migrate to create the respective tables in our database. We can now easily perfom CRUD operations for both nouns and verbs at http://0.0.0.0:3000/nouns and http://0.0.0.0:3000/verbs. We can automate the process of adding test data in our tables by using the Faker Gem and writing this code in our seeds.rb file.

seeds.rb

Loading Gist

This will create 500 dummy nouns and an additional 500 dummy verbs in our database. We can now populate our database by running

$ rake db:seed

Adding the after_save and before_destroy callbacks

We will then add an after_save and a before_destroy callback to our models (Noun and Verb). The after_save will be responsible for adding data to Redis through the Soulmate::Loader module provided by the soulmate gem and the before_destroy will remove data from redis when a record is deleted as we don’t want to have orphaned or unused records in our Redis database and also to save on Memory.

noun.rb

Loading Gist

verb.rb

Loading Gist

You will notice an optional data key. This is an optional container for metadata you'd like to return when an item is matched. For our case we add a link to the respective noun / verb url which will be useful later in that the search results will be clickable.

Loading data into Redis

Now that we have our model callbacks in place we can load all the data we ‘seeded’ earlier on into Redis by invoking a resave of each record which will trigger the load_into_soulmate callback in our model. Load rails console and execute the following

Make sure you have redis server running before you run this

$ rails console 2.1.0 :001 > Noun.find_each(&:save) 2.1.0 :001 > Verb.find_each(&:save)

This will re-save all records triggering the load_into_soulmate callback. Lets take a sneak peek at the data inside Redis and how it got saved by soulmate. On a separate terminal type in redis-cli to access the redis’ console.

$ redis-cli

We then query redis to fetch records in the soulmate-data hash of our first noun and verb respectively. You should get something similar to this

127.0.0.1:6379> hget soulmate-data:nouns 1 "{\"term\":\"alarm\",\"id\":1,\"data\":{\"link\":\"/nouns/1\"}} 127.0.0.1:6379> hget soulmate-data:verbs 1 "{\"term\":\"calculate\",\"id\":1,\"data\":{\"link\":\"/verbs/1\"}}"

The Redis key names use the pattern “ soulmate-data:nouns” and "soulmate-data:verbs" because I passed “nouns” and "verbs" as arguments when calling the Soulmate::Loader class.

Loading soulmate into our rails app

Soulmate can be loaded as a separate ‘web-server’ as its already a full sinatra app but for our case we want to have it inside rails. This can simply be done by adding it to our routes.rb file

routes.rb

Loading Gist

With that in place we can query soulmate at the /autocomplete url. Lets visit http://0.0.0.0:3000/autocomplete and see whats there. This should give you soulmate’s status

Now lets try querying some some data. We’ll start by querying some nouns. I will try querying the term pro and see what I got. Visiting http://0.0.0.0:3000/autocomplete/search?types[]=nouns&limit=6&term=pro give me

For the verbs, I will try searching any verbs starting with the characters co. Visiting http://0.0.0.0:3000/autocomplete/search?types[]=verbs&limit=6&term=co gives me

Waaaat!! That looks really Good. Don’t you agree? The interesting part and most powerful part of soulmate is that we can search all occurrences of a certain term across our models. To do this we need to append them to the types array in our url. This tells soulmate which results to include.

NOTE: THE TYPES YOU PASS IN THE URL MUST MATCH THE ONES YOU PASSED WHEN SAVING INTO SOULMATE (INSIDE OUR MODELS)

For our case we only have NOUNS and VERBS . To search both of them we make a call like this in the url http://0.0.0.0:3000/autocomplete/search?types[]=verbs&types[]=nouns&limit=6&term=ha . Note how we pushed the NOUNS and the VERBS inside the types array. My results will now include occurrences of “ha” in both nouns and verbs

Now that looks really great!! all we are now left with, is building the UI to utilize this awesome search functionality.

Piecing it all together using jquery and soulmate.js

Soulmate.js like we said earlier is an jQuery front-end for the soulmate auto-suggestion gem and together, they provide lightning-fast plug-n-play auto-complete / auto-suggestion. With soulmate.js we will provide our application with an auto-complete look similar to that at seatgeek.com.

Lets start by grabbing soulmate.js from github. Once downloaded grab jquery.soulmate.js located in soulmate.js-master/src/compiled folder and add it to the javascripts vendor directory of our rails app

//= require jquery //= require jquery_ujs //= require jquery.soulmate //= require turbolinks //= require_tree

Next we grab the demo.css stylesheet located in soulmate.js-master/demo and add it to the stylesheets folder in the vendor directory of our rails app. In this file we only need lines 30-115 you can do away with the rest as they were specific to the demo of its original author. We could also lets rename it to soulmate.css to keep the name semantic

*= require_tree . *= require soulmate *= require_self */

soulmate.css

Loading Gist

Adding a search form

Lets create a home controller and make it the default root path of our app. We will then put a search from in our index view of our home controller.

$ rails g controller home index

Make the root path point to our home's controller index action. In routes.rb append the following

root 'home#index'

Now lets add a search form in the index view of our home controller

index.html.erb

Loading Gist

Some quick simple styling

home.css.scss

Loading Gist

With this we should now have our 'fancy' home page with our search form

Our most awaited step comes next. Lets now power our search form with soulmate and soulmate.js goodness. Rename home.js.coffee in app/javascripts to home.js as we will be writing in pure javascript . In this file will call soulmate.js with our form search field. Remember we gave it an id of search.

home.js

Loading Gist

In the url option, we pass our soulmate search url we were using earlier. In the types option we pass an array of what we want to search from soulmate. If you have other models, add them here with the key you used to save into redis.

And finally lets test if this actually works . I'll start typing the work “ pa” and see what I get.

And that worked!!! Hurray! we now have a powerful lightening fast autocomplete. Play around with different search terms and see how fast and accurate the autocomplete is. Thats it for this tutorial and hope this gave you insight on how to come up with a fast autocomplete search in your rails applications.

Have any problems? Feel free to clone this application on github. For other questions and/or comments use the comments section below. Thanks for reading.

Update: This post was featured in Ruby Weekly. Thanks!

___