In a recent software quality audit, our static analysis tool Teamscale found that the comment completeness ratio of the application under study was 96%. But when examining these comments manually, we found that the majority of them was generated automatically and therefore of limited use.

This example illustrates that a combination of software tool and human expertise is necessary to get a holistic picture of and ensure software quality. To argue for this position in more detail, this blog post sketches software quality tasks which should be performed by software tools, software quality tasks which should be performed by human experts and software quality tasks which should be performed jointly.

Tool Results vs Reality

Software quality is an important aspect of a software application, especially for long-living software which is used and evolved over decades. To ensure software quality, much effort is put into quality activities like testing, bug fixing, code reviews, (static) quality analysis, and refactoring. Furthermore, many tools have been developed in the last years to automatically assess software quality and support developers to improve the quality of their application.

But often a software tool is introduced and induces a short-term improvement of software quality but has negligible positive effect on software quality in the long run. We argue that to achieve such a long-term positive effect, a combination of software tool and human expertise is necessary.

The example of a recent software quality audit demonstrates the interplay of software tool and human expert: When analyzing the Java source code of the application under study, our static analysis tool Teamscale found that the comment completeness ratio was 96%, i.e., that 96% of all public types, attributes and methods were commented. This appears to be a very good value and would rank the application very high in our benchmark of industry software.

But when we manually analyzed a subset of these comments, we found that many of them were trivial comments that were generated from identifiers in code in order to adhere to the company policy »More than 95% of types, attributes and methods have to be commented«. The following code listing shows an example of such a trivial comment: