1. Fast

One of the major benefits of writing unit tests is it allows you to document how your code should work (as well as prove it actually works). The only way you can take advantage of that benefit is if your tests are fast. Engineers are impatient (as most people are), if it takes more than a second to verify your code works there is little chance you will want to run them with any sort of frequency. Running tests frequently is crucial to finding regressions in your code, the longer you wait, the more time you’ll spend figuring out why a test is failing.

Making sure your unit tests are fast isn’t something I cared about until I had to wait 15 minutes for 2,000 unit tests to execute. While writing tests for Android I’ve discovered a few things to avoid if you want to have fast tests. The first thing you should do is say no to Robolectric. While Robolectric has it’s benefits, the last time I used it in a project I spent 30 seconds waiting for it to initialize in most tests. If you want to use Robolectric you should ensure it runs separately from your actual unit test suite.

The second thing you should do is limit your reliance on things like PowerMock since preparing a static function can take upwards of one second. In most cases you can avoid using PowerMock by writing code that doesn’t rely heavily on statics.

The third is only use Mockito when you actually need to, using it to mock a data class is probably overkill. However if you have some complex business logic you need to mock (such as a UseCase) then it’s probably worth using a mocking framework.

Just for reference, the last CircleCI build we had for our Android app at ActiveCampaign reported that our 246 unit tests took a total of 1.501 seconds to run. While we don’t have that many unit tests currently (hey, we’re only 10 weeks into this project), when we do reach 2,000 unit tests it should only take about 12 seconds to run the entire suite. We have been able to achieve these numbers by avoiding Robolectric and PowerMock as well as only using Mockito when we absolutely need to.

2. Independent

Yet another mistake I’ve seen are tests that are not indepdent of one another. When I say independent I mean two things: your tests only test one thing and your tests do not rely on one another to pass. So how can you avoid falling into these mistakes?

For testing one thing you should be able to write a sentence that explains what your test is doing and what you expect it to return. If you find yourself adding and to the sentence’s expected behavior portion you’re probably test is probably too large. Another way to know is if you are calling the function your testing more than once and asserting on the different outputs. Here’s an example of a good unit test:

@Test

fun `string with http will remove http when calling extractSubDomain`() {

val given = "http://activecampaign" assertThat(given.extractSubDomain()).isEqualTo("activecampaign")

}

This unit test is good because it’s only testing one aspect of the function which is whether or not it will remove http:// from a string. If something changes in our extractSubDomain function that prevents this from working correctly this test will fail however other tests around the same function should still pass. By making these tests independent of each other we are able to better document everything that we expect to happen while also knowing what exactly broke. So what does a bad unit test look like?

@Test

fun extractSubDomain_WorksCorrectly() {

var given = "http://activecampaign"

assertThat(given.extractSubDomain()).isEqualTo("activecampaign") given = "activecampaign.com"

assertThat(given.extractSubDomain()).isEqualTo("activecampaign")

}

In this example we are updating our given value and trying to assert on two different inputs. This is bad for a number of reasons, but in terms of independent our second assertion is now relying on the first assertion to succeed. This means that if the code to remove the TLD works correctly but the code to remove the protocol does not we’ll only know that the latter is broken. This may not seem like a big deal, but when you have a fix failing unit tests it’s better to know up front exactly what is broken as opposed to only knowing about the first thing that fails.

The last thing to point out when it comes to independent tests is the way that test runners typically work. In general if you use JUnit you will probably have a hard time writing tests that rely on one another since they inherently will run randomly. Actually, it’s more likely that you’ll accidentally write tests that rely on one another which will surface as flakey tests out in the wild. Regardless though, in this case the testing framework is doing all of the work for you, so unless you have singletons that are seeding test data you should be safe.

3. Thorough

It doesn’t matter how good your unit tests are if you hardly write them. If you are writing independent tests then being thorough is a fairly easy next step. While writing your unit tests you should at a minimum ensure all of the possible outputs are covered for a given input. In other-words for every condition in your function you should have at least one unit test that asserts it works correctly. In our example for independent tests we had a function called extractSubDomain , so let’s expand on that further by showing what the actual function looks like.

fun String.extractSubDomain(): String {

return replaceBefore("/", "")

.replace("/", "")

.replaceAfter(".", "")

.replace(".", "")

}

This probably isn’t what you’d expect to see as the solution may seem like it’s made for RegEx (in the future maybe it will be). The body of this function is just four lines and yet we have a total of eight tests to ensure that this works correctly. Here are the names of those tests:

@Test

fun `string without any domain characteristics is left unchanged when calling extractSubDomain`() {}



@Test

fun `string with http will remove http when calling extractSubDomain`() {}



@Test

fun `string with https will remove https when calling extractSubDomain`() {}



@Test

fun `string with tld will remove tld when calling extractSubDomain`() {}



@Test

fun `string with domain name will remove domain name when calling extractSubDomain`() {}



@Test

fun `string with tld and http will remove everything except the subdomain when calling extractSubDomain`() {

//note: and is okay in this case as it's describing the input

}



@Test

fun `string with domain name and http will remove everything except subdomain when calling extractSubDomain`() {}



@Test

fun `string with domain name and https will remove everything except subdomain when calling extractSubDomain`() {}

This many tests is necessary because the subdomain could be http://medium.activecampaign.com , http://medium , medium.activecampaign.com , etc. So simply testing that we remove one piece or another isn’t enough as we also need to ensure thing a complete domain protocol, domain, and TLD can still be removed so the sub domain can be extracted. This of course is not perfect, you may notice that activecampaign.com will result in activecampaign being returned; for the problem we are solving though it is actually adequate. When the solution is no longer enough we can make extractSubDomain() more robust while ensuring the past functionality still works.

On the other-hand, if we weren’t thorough with these unit tests we could have inadvertently added blind spots to our code. Those blind spots can result in regressions cropping up without the test suite failing. So when it comes to writing good unit tests, make sure you’re thorough. Anytime a regression slips through into production it’s usually a sign that your test suite was not as thorough as it could have been (although this is fine so long as you write the test to prove the bug exists and then fix the code).