You probably found this article by searching if you can secure your Elasticsearch cluster by providing some sort of authentication mechanism. You might have come across many options like Shield, or some of the open-source authentication plug-ins available in Qbox, but you decided that you want to have some fun by creating your own plugin for authenticating the REST requests coming to Elasticsearch.

In this tutorial we explain how you can plug-in basic authentication into Elasticsearch. This article is based on the APIs available in Elasticsearch 2.3.x.

Tutorial

For this post, we will be using hosted Elasticsearch on Qbox.io. You can sign up or launch your cluster here, or click "Get Started" in the header navigation. If you need help setting up, refer to "Provisioning a Qbox Elasticsearch Cluster."

Finding where to hook the plugin

There are several points in Elasticsearch where you can hook your plugin. We will not be going through all of them. Our primary focus is on the authenticating the REST requests made to communicate to Elasticsearch. There is RestFilter class in Elasticsearch which filters all the REST requests coming to it. We will be extending the RestFilter class to implement the authentication logic. In the constructor, you will have to register your class so that the RestController is aware of this class and will divert the traffic through it.

What methods to override?

The RestFilter class has two methods that we have to override in our class. The process() method which processes the rest request, and the order() method which decides the order of execution of the RestFilter classes. The process() method can be used to invoke the authentication logic that you are going to implement or already have in place. Once the authentication is completed within the process() method, the filter must continue to the rest of the operations intended by the request. For that, at the end of the process() method, you will have to call the continueProcessing() method of RestFilterChain.

How do I get my filter class to run first?

The order of execution of the filter is important because our filter should be the first one to get executed, and that is why we need to return the order number in the order() method. By default, it will be zero and the execution is done from lowest value to highest. Since the lowest value of an integer will be -2147483648, it is better to return this value instead of default zero.

How to authenticate?

You may be familiar with all the varieties of authentication mechanisms like Basic, Digest, OAuth, etc. To keep things simple, we will be discussing only Basic Authentication. In the case of Basic Authentication, the client sends out the username and password as Base64 encoded text in the Authorization header of the request. We just have to extract the encoded text, decode it and see if the user is authorized to access Elasticsearch. To know if user is authorized, we need to keep a record of the users and password in the Elasticsearch side. You can either add it in the elasticsearch.yml file or find some other ways to keep a list of users. Here we will just put it in elasticsearch.yml file as http.user and http.password . If the header is not present in the request a challenge will be sent back to get the Authorization header.

When the filter class is completed, it will be something like:

public class myCustomRestFilter extends RestFilter{ public myCustomRestFilter(RestController controller, Settings settings){ this.settings = settings; controller.registerFilter(this); } public void process(RestRequest request, RestChannel channel, RestFilterChain filterChain) { if (request.header("Authorization") == null) { RestResponse response = new BytesRestResponse(RestStatus.UNAUTHORIZED, "Access denied"); response.addHeader("WWW-Authenticate", "BASIC realm=\"Elasticsearch Server\""); channel.sendResponse(response); } else{ String encoded = request.header("Authorization").replaceFirst("Basic", "").trim(); String[] credentials = Base64.decode(encoded).split(":", 2); String user = settings.get("http.user", "adminuser"); String password = settings.get("http.password", "iamtheadmin"); if (credential[1] == user && credential[2] == password){ filterChain.continueProcessing(request, channel); } else{ Throw NotAuthorizedException(“User not authorized”) } } } public int order() { return -2147483648; } }

Conclusion

We have seen how to make use of RestFilter to add authentication into Elasticsearch. Now all we need to do is plug this into Elasticsearch’s plugin module and run it. Plugin development is another topic that we will discuss in our upcoming article. See you soon!

Related Helpful Resources

Give It a Whirl!

It's easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, Amazon or Microsoft Azure data centers. And you can now provision a replicated cluster.

Questions? Drop us a note, and we'll get you a prompt response.

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.