Elasticsearch is a search engine technology to implement powerful and fast search features. Like every additional component of your Java application Elasticsearch should be covered by integration tests. In this blogpost I show how tests can be written using alexcojocarus Maven plugin.

Ways to write tests

There are many ways to write tests for your Java application that uses Elasticsearch. First of all there is a Java Testing Framework offered by the elastic team themselves [4]. Additionally there is the possibility to use an ant script to download and run Elasticsearch and test against it [2].

I decided to use the Maven plugin developed by alexcojocaru [1]. The plugin downloads and starts an Elasticsearch instance. This enables you to run your tests on an actually running Elasticsearch server.

The plugin is easy to set up and powerful enough to suit my needs. Additionally it is not too tightly integrated with the search engine like Elasticsearchs own testing framework. This ensures that the tests will behave just like in a real productive environment.

Setting things up

I decided to name all my Elasticsearch based test classes with an „ESTest“ suffix. I added that pattern to the surefire plugin of the project:



<plugin>

<artifactId>maven-surefire-plugin</artifactId>

<version>2.18.1</version>

<configuration>

<includes>

<include>**/*IntTest.java</include>

<include>**/*UnitTest.java</include>

<include>**/*ESTest.java</include>

</includes>

</configuration>

</plugin> 1 2 3 4 5 6 7 8 9 10 11 <plugin> <artifactId> maven-surefire-plugin </artifactId> <version> 2.18.1 </version> <configuration> <includes> <include> **/*IntTest.java </include> <include> **/*UnitTest.java </include> <include> **/*ESTest.java </include> </includes> </configuration> </plugin>

Setting up and configuring the plugin is quite easy. In the configuration section you can define the clusterName and the version. Mind that the plugin doesn’t allow special characters. The version should be the same as Elasticsearch has in in your application.

The default port numbers are 9200 for the HTTP port and 9300 for the transport port. I recommend using different port numbers to avoid conflicts with your local Elasticsearch installation (I used 9400 and 9500).

In the executions section you can define in which phases the plugin gets started or stopped. I decided to use the process-test-classes and the prepare-package phases so I can run my tests in the test phase (which is just right between these phases).



<plugin>

<groupId>com.github.alexcojocaru</groupId>

<artifactId>elasticsearch-maven-plugin</artifactId>

<version>5.3</version>

<configuration>

<clusterName>testCluster</clusterName>

<transportPort>9500</transportPort>

<httpPort>9400</httpPort>

<version>5.1.2</version>

<pathInitScript> src/test/resources/elasticsearch/Init.script</pathInitScript>

</configuration>

<executions>

<execution>

<id>start-elasticsearch</id>

<phase>process-test-classes</phase>

<goals>

<goal>runforked</goal>

</goals>

</execution>

<execution>

<id>stop-elasticsearch</id>

<phase>prepare-package</phase>

<goals>

<goal>stop</goal>

</goals>

</execution>

</executions>

</plugin> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 <plugin> <groupId> com.github.alexcojocaru </groupId> <artifactId> elasticsearch-maven-plugin </artifactId> <version> 5.3 </version> <configuration> <clusterName> testCluster </clusterName> <transportPort> 9500 </transportPort> <httpPort> 9400 </httpPort> <version> 5.1.2 </version> <pathInitScript> src/test/resources/elasticsearch/Init.script </pathInitScript> </configuration> <executions> <execution> <id> start-elasticsearch </id> <phase> process-test-classes </phase> <goals> <goal> runforked </goal> </goals> </execution> <execution> <id> stop-elasticsearch </id> <phase> prepare-package </phase> <goals> <goal> stop </goal> </goals> </execution> </executions> </plugin>

When running your tests (mvn clean verify) you should see that Elasticsearch gets started.

For your tests you should also add junit to your Maven dependencies.

Straightening things out

Setting up the Maven plugin worked mostly fine for me. However I had some heap size problems at first which prevented Elasticsearch from starting up. I had to make sure the ES_JAVA_OPTS environment variable is set before I run the tests:



export ES_JAVA_OPTS="-Xms1g -Xmx1g" 1 export ES_JAVA_OPTS = "-Xms1g -Xmx1g"

Another problem you might face while writing tests is that when errors occur the next clean up face of Maven will fail because the process was not shut down. In that case the process can be killed. The pid is stored in a file in the targetelasticsearch<instance> folder.

To kill the process use this command for windows:



taskkill /F /pid <pid> 1 taskkill / F / pid < pid >

or this command for unix systems:



kill -9 <pid> 1 kill - 9 < pid >

Preparing your data

When writing your tests you could of course use Elasticsearchs Java API to create your index and fill it with test data. The Elasticsearch Maven plugin offers init scripts to prepare your cluster which makes things easier. The path to the initScript can be defined in the configuration section of the plugin:



<pathInitScript>src/test/resources/elasticsearch/Init.script</pathInitScript> 1 <pathInitScript> src/test/resources/elasticsearch/Init.script </pathInitScript>

I then created an Init.script file in the test/resources path of my directory.

A simple script file to create an index and insert a document could look like this:



#create mapping

PUT:myIndex/:{ "mappings":{"default":{ "properties":{ "myField":{"type":"keyword"}}}}}

#add document

PUT:myIndex/default/1:{ "myField ":"hello world"} 1 2 3 4 # create mapping PUT : myIndex / : { "mappings" : { "default" : { "properties" : { "myField" : { "type" : "keyword" } } } } } # add document PUT: myIndex / default / 1 : { "myField " : "hello world" }

You can use all commands Elastcisearchs REST API offers. The syntax consists of the request method, the path and the JSON document separated by colons. The JSON document can be omitted when it is not necessary (for example for deletes).

Comment lines start with „#“.

Writing your tests

Now that you have an Elasticsearch cluster with your indices and some test data you can start with your tests. I wrote an abstract class to initialize and close the transportClient before and after the tests. The class can later be used as an upper class for your test classes.



public abstract class InitializeESTesting { private final static String ADDRESS = "127.0.0.1";

private final static int PORT = 9500;

private final static String CLUSTER_NAME = "testCluster";

protected final static String INDEX_NAME = "myIndex"; protected static TransportClient transportClient; protected static final Logger LOGGER = LoggerFactory.getLogger(InitializeESTesting.class); @BeforeClass

public static void setupTransportClient() {

Settings settings = Settings.builder().put("cluster.name", CLUSTER_NAME).build();

transportClient = new PreBuiltTransportClient(settings);

try {

transportClient.addTransportAddress(

new InetSocketTransportAddress(InetAddress.getByName(ADDRESS), PORT)

);

} catch (UnknownHostException ex) {

LOGGER.error("Couldn't add transport address" + ex);

}

} @AfterClass

public static void setupTransportClient() {

if (transportClient != null) {

transportClient.close();

transportClient = null;

}

} 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 public abstract class InitializeESTesting { private final static String ADDRESS = "127.0.0.1" ; private final static int PORT = 9500 ; private final static String CLUSTER_NAME = "testCluster" ; protected final static String INDEX_NAME = "myIndex" ; protected static TransportClient transportClient ; protected static final Logger LOGGER = LoggerFactory . getLogger ( InitializeESTesting . class ) ; @BeforeClass public static void setupTransportClient ( ) { Settings settings = Settings . builder ( ) . put ( "cluster.name" , CLUSTER_NAME ) . build ( ) ; transportClient = new PreBuiltTransportClient ( settings ) ; try { transportClient . addTransportAddress ( new InetSocketTransportAddress ( InetAddress . getByName ( ADDRESS ) , PORT ) ) ; } catch ( UnknownHostException ex ) { LOGGER . error ( "Couldn't add transport address" + ex ) ; } } @AfterClass public static void setupTransportClient ( ) { if ( transportClient != null ) { transportClient . close ( ) ; transportClient = null ; } }

I am now able to write tests with classes extending the InitializeESTesting class and using the *ESTest name scheme.

My first two simple tests check if the index „myIndex“ I created with the initScript was created successfully and if there is exactly one document in my index.

With the transportClient successfully established all kinds of queries can now be executed.



public class MyESTest extends InitializeESTesting {

@Test

public void isMyIndexIndexSetUp() {

assertTrue(transportClient

.admin()

.indices()

.exists(new IndicesExistsRequest("myIndex"))

.actionGet()

.isExists());

} @Test

public void checkMessageIndexSize() {

IndicesStatsResponse response = transportClient.admin().indices().prepareStats(INDEX_NAME).execute().actionGet();

long docCount = response.getIndex(INDEX_NAME).getTotal().getDocs().getCount();

assertEquals(docCount,1L);

} } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 public class MyESTest extends InitializeESTesting { @ Test public void isMyIndexIndexSetUp ( ) { assertTrue ( transportClient . admin ( ) . indices ( ) . exists ( new IndicesExistsRequest ( "myIndex" ) ) . actionGet ( ) . isExists ( ) ) ; } @ Test public void checkMessageIndexSize ( ) { IndicesStatsResponse response = transportClient . admin ( ) . indices ( ) . prepareStats ( INDEX_NAME ) . execute ( ) . actionGet ( ) ; long docCount = response . getIndex ( INDEX_NAME ) . getTotal ( ) . getDocs ( ) . getCount ( ) ; assertEquals ( docCount , 1L ) ; } }

With mvn clean verify the tests should now run successfully. The test results can be found in the surefire-reports target folder.

Useful Links