Earlier this year I wrote an article on how to run your integration tests with an embedded elasticsearch. When upgrading to elasticsearch 7 this method didn’t work (yet). An alternative (and maybe even better) method is using Testcontainers to run elasticsearch in a Docker container. I will also show how you can leverage Karate to do your integration testing.

Setup and requirements

For this article I use Maven 3.6.1, Java 11 and Testcontainers (a Maven dependency).

Testcontainers uses Docker, so you need to have Docker installed.

Note that when you eventually run Testcontainers on Jenkins you need to be able to run Docker on Jenkins too.

To verify Maven and Java are installed correctly:

mvn -v

Should contain a Maven and Java version.

To verify Docker is installed and running correctly:

docker info

Should contain a result like : Server Version: 18.09.2

All the sample code is available at github and I will show important code snippets in this article.

Use Karate to test elasticsearch – You know, for search

Our initial step is a JUnit test that spins up a testcontainer with elasticsearch and a Karate-test that checks if elasticsearch is running.

Add a Maven dependency

To use the elasticsearch Testcontainer add the following Maven dependency :

<dependency> <groupId>org.testcontainers</groupId> <artifactId>elasticsearch</artifactId> <version>1.11.4</version> </dependency>

Create your test class

Add a class DemoKarateIT :

@Testcontainers public class DemoKarateIT { private static final String url = "docker.elastic.co/elasticsearch/elasticsearch:7.2.0"; @Container private static ElasticsearchContainer container = new ElasticsearchContainer(url); @BeforeAll static void beforeAll() { String httpHostAddress = container.getHttpHostAddress(); System.setProperty("elasticsearch.address", httpHostAddress); System.setProperty("karate.env", "test"); } @Karate.Test Karate testAll() { return new Karate().feature("classpath:karate"); } }

Karate uses system properties to pass values. karate.env is used to determine whether you’re running the tests from JUnit or standalone (more on this later). The most simple use of a Testcontainer is annotating a container field with @Container and adding the @Testcontainers annotation to your test. We also call the container.getHttpHostAddress method to pass the address via the elasticsearch.address . Note that the port is randomized, so we can’t hardcode the port number.

Using this method a container is started/stopped on every method. When you only want to start the container once use the singleton container pattern (thanks to Sergei Eigorov for pointing this out).

Add the Karate config

The karate-config.js :

(function () { var env = karate.env; // get java system property 'karate.env' if (!env) { env = 'local'; //no environment found, assuming local (laptop) karate.properties['elasticsearch.address'] = 'localhost:9200' } karate.log('karate.env property:', env); var config = {}; return config; });

The check on env is added so we can run the .feature files directly from within the IDE without JUnit (you need a running elasticsearch instance (and possibly other services) for this) which can speed up development. When the env is set (in our unit-test to ‘test’) the address of the elasticsearch Docker container is used.

Add a Feature file

Finally we have to add a feature file called es-basic.feature :

Feature: Basic elasticsearch tests Background: Scenario: Verify elasticsearch is running Given url 'http://' + karate.properties['elasticsearch.address'] + '/' When method GET Then status 200 And match $.tagline == 'You Know, for Search'

This test will call the root url and verifies the reason where you want to use elasticsearch for.

Now you can run the integration test DemoKarateIT. Note that the first time you run this test it might take a while since the Docker image for elasticsearch needs to be downloaded once. This is a bit of a caveat on Jenkins, if configured incorrectly the container will be downloaded every time.

Auto detect the elasticsearch version

For my previous article I used a class AutoDetectElasticVersion to auto-detect the version of elasticsearch. You can use this class to use the same Docker image version as your Maven elasticsearch dependency.

Prepare your test data

Now that our basic setup is running it is time to create some test data to create some realistic tests.

It’s imperative to have proper test data in your project. When there is no proper data try to create realistic data (not just data to keep your tests happy), this will prevent many nasty bugs.

In the sample code there is a class called OwnerGeneratorUtil that produces json lines ( .jsonl ) that can be used to do bulk inserts in elasticsearch.

A sample record :

{"index":{"_index":"owner","_id":"ee38304188"}} {"id":"ee38304188","firstName":"Jane","lastName":"Doe","car":"BMW"}

Note that the .jsonl file should have an empty line at the end.

The background step in a Karate Feature is executed on every Scenario. When you’re doing read only requests on elasticsearch it makes no sense to re-insert the data every time. It also prevents you from running your tests in parallel (to disable parallel execution add the tag @parallel=fasle to your feature). To do an insert only once you can use callonce

In our example the callonce reads a feature file and executes it with the file names as parameter :

* callonce read('es-load-testdata.feature') { 'fileName' : 'test-owners.jsonl' }

It is important when loading data in elasticsearch to call refresh afterwards since the data might not be available for search yet!

Creating a real test

For this test we spin up a Vert.x server that exposes an endpoint at /all and /{search term} .

It is not really important that we user Vert.x here, it just is more realistic than calling elasticsearch endpoints directly.

es-advanced.feature

Feature: More advanced elasticsearch tests Background: * url baseUrl * callonce read('es-load-testdata.feature') { 'fileName' : 'test-owners.jsonl' } Scenario: Verify find all Given path '/all' When method GET Then status 200 And match $ == '#[180]' Scenario: Verify Hans Gruber drives a Peugeot (might return other Grubers) Given path '/gruber' When method GET Then status 200 And match $ contains any { 'firstName' : 'Hans', 'lastName' : 'Gruber', 'car' : 'Peugeot', 'id' : '#notnull' }

The first scenario verifies that there are 180 records when calling the /all endpoint.

The second scenario searches for ‘gruber’ and verifies that Hans Gruber drives a Peugeot (who would’ve guessed that?). Not that you can use the #notnull placeholder when handling generated id’s (note that this is technically a javascript String).

Miscellaneous

In this paragraph I will tie up some loose ends you also might run into.

JUnit 5 with Karate

To use Karate with JUnit 5 you have to add a dependency :

<dependency> <groupId>com.intuit.karate</groupId> <artifactId>karate-junit5</artifactId> <version>0.9.4</version> <scope>test</scope> </dependency>

Running tests in parallel with Karate and JUnit 5

When you want to run your tests in parallel you can’t use the @Karate.Test annotation anymore. Use the following snippet instead:

@Test void testParallel() { Results results = Runner.parallel(getClass(), 2, "target/failsafe-reports"); assertThat(results.getFailCount() == 0).isTrue().withFailMessage(results.getErrorMessages()); }

This method will start 2 threads and won’t look as nice when running it in your IDE (with @Karate.Test every scenario was displayed separately, but that’s probably impossible or very challenging).

Use the tag @parallel=false to prevent Scenario’s to run in parallel (Features will still run parallel though).

Surefire, failsafe or skip everything?

Since the JUnit test is actually an integration test the class name should end with IT so it will be ran with the failsafe plugin instead of the surefire plugin.

Don’t forget to add the proper goals and use a least version 2.20.0 to prevent issues :

<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.22.2</version> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> </plugin>

mvn clean verify -Dmaven.test.skip will skip all tests (where -DskipTests will only skip unit tests). Skipping tests is frowned upon so try to improve run times of your tests instead of skipping them.

Conclusion

I hope this was a coherent piece to read. I wanted to include a lot of issues you might run into to save you the troubleshooting. If you have any comments/improvements please let me know. Thanks for reading!

Sources and more information

Karate

Testcontainers (elasticsearch module)