Case 2: Your coverage is below the Threshold

Keeping in mind that you know the importance of a reasonable test suite and actually want to write all the necessary tests. Let’s have a look at the second situation where the actual coverage is below that arbitrary threshold even though you have written all the reasonable tests.

I can think of two possible options you have to still reach the threshold.

Excluding new code from the coverage report. Writing more tests.

Option 1: Excluding code from the coverage report:

This option is only applicable when you are in control over the coverage runner.

Some of the code coverage runners allow the user to exclude certain parts of the code so they are not counted towards the overall coverage result.

Many of these runners however, only allow whole files/packages to be excluded. Given that a class might have code that needs to be tested and code that does not need to be tested (e.g. hashCode() which you have auto-generated and only have because of equals() ) you may end up excluding code that you actually want to be tested. The coverage report will, in this case, not call out when there is untested code in that file/package.

This is again the exact opposite of what the person who set up the threshold tries to achieve.

Option 2: Writing more tests:

As mentioned above I assume you value a sufficient test suit and you already wrote all the tests you considered reasonable. But when you now still need to increase your code coverage to reach that threshold the only thing left is writing unreasonable tests.

This simply is a waste of time, effort and money and surely not what your management has in mind.

At the end it boils down to Goodhart’s law, which states:

When a measure becomes a target, it ceases to be a good measure.

After all, mandating the developers to achieve a certain threshold of code coverage looks like a sign of not trusting a development team.

In times where self-empowered teams are considered a key to success of software development projects, this approach seems to be going in the opposite direction.

If you manage a development team and feel that not enough tests are being written, I’d suggest, rather than imposing counterproductive measures, enable your team. Spend some money on developer training about tests — you’ll get it back manifold when the developers understand the real benefits of a reasonable test suite.

Summary: