The fundamental problem is that unit testing is hard to do right. It needs to be a part of software design, not coding and you should never write your tests to your code (but everybody does this anyway).

Note, Unit tests are great. However, I recognise that the basic test cases really are “software engineering” rather than “development territory”. So this whole argument points at a much deeper problem, and one that test driven development is the answer for.

Python Midnight Time

Let’s draw a parallel to the Python midnight time as boolean discussion for a moment. The Python midnight time issue was entirely because the developer of the library made his code the contract.

“This is how it works so hence live with what that means” is not really a contract and in that case it had some really nasty corner cases (time-zones west of GMT never evaluate to false for any time with a timezone, but time-zones UTC and east evaluate to false when the time represents UTC midnight).

But a lot of people didn’t want to break that contract. Really? That’s a contract that could only be appreciated by tax lawyers.

Code is the contract

The same fundamental problem happens with unit tests. “This code works so it must be correct” is a typical response and so the code becomes the contract. A bug is found, and fixed, and the test cases start failing. This is a waste of time. It is a waste of time to write the test. It is a waste of time to remove it because it is irrelevant.

Do extensive unit testing, however, take the unit tests seriously. Before you write a test, ask yourself “If a bug is found and fixed in the software or any of its dependencies, should this test ever fail?” If the answer is “yes” then don’t write the test.

Test is the contract

The point of unit tests are to ensure that software contracts are not mistakenly broken. This is true no matter how you write code. Test scripts are mostly longer code-wise than the production code. This is true for SQL and PL/PGSQL, just as it is for Perl.

However, try hard to ensure that a test failure means that a previously working API is no longer working rather than that you changed something.