$\begingroup$

I'm not 100% sure if the explanation below is historically accurate. If it isn't, please feel free to edit or remove.

Mutation testing was invented by Lipton. Mutation testing can be seen as a way to measure the quality or effectiveness of a test suite. The key idea is to inject faults into the program to be tested (i.e. to mutate the program), preferably the kinds of faults a human programmer is likely to make, and see if the test suite finds the introduced faults. A typical example of the kind of fault mutation testing would introduce could be to replace x > 0 by x < 0, or replace x by x+1 or x-1. The fraction of faults caught by the test suite is the "mutation adequacy score" of a test suite. Speaking very loosely, one can think of this as a Monte-Carlo method for computing the mutation adequacy score.

More abstractly one might say that mutation testing brings to the fore a symmetry or duality between a program and its test suites: not only can the test suite be used to become more confident about the correctness of a program, but conversely, a program can be used to gain confidence about the quality of a test suite.

In the light of this duality, mutation testing is also conceptually close to fault injection. Both are technically similar but have different purposes. Mutation testing seeks to measure the quality of the test suite, while fault injection seeks to establish the quality of the program, usually the quality of its error handling.

Recently, ideas from mutation testing have been used to test (formalisations of) logical theories. To paraphrase the abstract of (4): When developing non-trivial formalizations in a theorem prover, a considerable amount of time is devoted to “debugging” speciﬁcations and theorems. Typically, incorrect speciﬁcations or theorems are discovered during failed proof attempts. This is an expensive form of debugging. Therefore it is often useful to test conjectures before embarking on a proof. A possible way of doing this is to assign random values to the free variables of the conjecture and then evaluate it. (4) uses mutations to test the quality of the used test-case generators.

History. From (1): The history of mutation testing can be traced back to 1971 in a student paper by Richard Lipton [...] The birth of the field can also be identified in other papers published in the late 1970s by Lipton et al. (2) as well as Hamlet (3).