Writing tests is crucial. Sometimes it’s difficult, sometimes boring, but it is needed to prove that the system works correctly. When an application includes a database, queue or other dependencies, then you need to write integration tests, confirming that integration with those systems is done properly. You need to check that APIs are used correctly or that the SQL queries are valid. However, to do that, you need to have those dependencies available during tests. How? Let’s discuss this today.

Before and after

A quite common case you may see is to use embedded versions of your dependencies, that is e.g. embedded Cassandra or Kafka or an in-memory database as an alternative to a standard one. There are a few challenges — before tests are run, they need to be started and after tests are finished, they need to be shut down. Usually you achieve that by starting infrastructure in tests Before type methods. It’s even easier to start before each test and shut down after each one. However, you need to remember two things:

Time — often it takes time to launch an embedded database. The more you start and stop it, the longer the tests take. If you start and stop around each test case, then the whole project test phase may take ages. That is why it's common to start in BeforeAll -like and stop in AfterAll -like methods. However, this has some consequences. State — database has its state and if it's started just once, before all tests in the test suite, then the further tests have to deal with a dirty state. There are some workarounds for that, which works for databases, but not necessarily e.g. for queues:

to drop & recreate schema after each test,

to truncate data after each test,

to rollback transactions,

to make tests independent by using different entity ids, and for tests of methods like getAll check how the state changed during the test, not asserting the whole response content.

Let’s take a look how this may look like in practice in Java with JUnit 4 ( @Before and @After or @BeforeClass and @AfterClass )

Junit 4

or leveraging JUnit 4 Rules ( @Rule or @ClassRule ):

JUnit 4 Rule

or with JUnit 5 ( @BeforeEach and @AfterEach or @BeforeAll and @AfterAll ):

JUnit 5

or with Scala and Scalatest ( BeforeAndAfterEach or BeforeAndAfterAll )

Scala with Scalatest

JUnit 5 has also an equivalent of JUnit 4 rules, based on @ExtendWith annotation, however, there is no built-in class for that in cassandra-unit .

In practice the quality of embedded versions of databases and queues is different. In the past, we have met cases of memory leaks or some processes not being cleanly shut down. Additionally, you may get dependency conflicts —e.g. databases are usually a quite complicated projects, so they include a lot of dependencies, which may conflict with versions of ones used directly by your project.

With Java, actually you have one more choice — to use the Testcontainers project. It allows you to easily integrate Docker containers with the execution of your tests. For example:

JUnit 4

Given piece of code would start a whole Cassandra container around each test case. Of course, you don’t need to leverage only generic classes, because Testcontainers modules support quite a big number of various databases and servers. If you’d like to make to ‘end-to-end’ type tests, then you can even run a Docker Compose .yml .

Warning: we have noticed in our tests that Testcontainers is redownloading images for each test file. No image caches were used. There is an issue related to that topic, improvements were already merged but not yet released.

Test suites

To improve performance some people are using test suites. Those are the cases, where you can annotate a dedicated class with a list of other test classes. In practice, tests are being grouped, so you can run a database before a whole bunch of tests.

Drawbacks? You can’t just run a single test, because a database would not be started. If you want to do that, you need to start it separately. This makes local development/debugging more troublesome.

JUnit 4

Dependencies outside

Another approach is to just start dependencies outside, before running tests. You can start them natively or in containers. We usually keep Docker compose scripts with our projects. They contain all the things needed to run the service. When new person joins, they need to run only two things — the mentioned docker-compose and the application with the default settings. The same scripts can be leveraged for tests, just with different schema names (so that tests wouldn’t clean the state you maintain for local service running), queues or topics. This way, the database starts once, before all tests, so overhead is lower. However, this brings other challenges. You need to remember about the state, but also you need to start it on the CI, before your tests. Sometimes docker is not available on CI machine and sometimes you have to deal with ports conflicts between different executions which may run in parallel. This definitely requires a bigger effort from DevOps. Fortunately various CIs today have built-in docker support — e.g. CircleCI or Google Cloud Build.

Conclusions

Tests should be fast in order to keep people running them locally. There is no perfect approach to starting external dependencies of your tests. You need to decide, what is more convenient to you and if your CI handles the designed flow. Remember that, in most solutions, you’re not limited only to JUnit, but often you may be able to leverage Spock or other test frameworks also from other languages, as Scala.