Mutation testing is conceptually quite simple.

Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived.

The quality of your tests can be gauged from the percentage of mutations killed.

What?

To put it another way - PIT runs your unit tests against automatically modified versions of your application code. When the application code changes, it should produce different results and cause the unit tests to fail. If a unit test does not fail in this situation, it may indicate an issue with the test suite.

Why?

Traditional test coverage (i.e line, statement, branch, etc.) measures only which code is executed by your tests. It does not check that your tests are actually able to detect faults in the executed code. It is therefore only able to identify code that is definitely not tested.

The most extreme examples of the problem are tests with no assertions. Fortunately these are uncommon in most code bases. Much more common is code that is only partially tested by its suite. A suite that only partially tests code can still execute all its branches (examples).

As it is actually able to detect whether each statement is meaningfully tested, mutation testing is the gold standard against which all other types of coverage are measured.

Why PIT?

There are other mutation testing systems for Java, but they are not widely used.

They are mostly slow, difficult to use and written to meet the needs of academic research rather than real development teams.

PIT is different. It's

fast - can analyse in minutes what would take earlier systems days

- can analyse in what would take earlier systems easy to use - works with ant, maven, gradle and others

- works with ant, maven, gradle and others actively developed

actively supported

The reports produced by PIT are in an easy to read format combining line coverage and mutation coverage information.

Example snippet taken from coverage report of Wicket Core

Light green shows line coverage, dark green shows mutation coverage.

Light pink show lack of line coverage, dark pink shows lack of mutation coverage.