TITEL

A history guide to software bugs

SPREKERS

Cathal Boogerd, promovendus, TU Delft

Martin Pinzger, universitair docent, TU Delft

TAAL

Nederlands of Engels, sheets in het Engels

ABSTRACT

Coding standards have become increasingly popular as a means to ensure software quality throughout the development process. They typically ensure a common style of programming, which increases maintainability, and prevent the use of potentially problematic constructs, thereby increasing reliability. Over the years various tools have become available that automate the checking of rules in a standard, helping developers to locate potentially difficult or problematic areas in the source code. In a recent investigation of bug characteristics, it was argued that early automated checking has contributed to the sharp decline in memory errors present in software.

However, automated inspection tools are notorious for producing an overload of non-conformance warnings (referred to here as violations). It has been observed that all tools suffer from incorrect violations or false positives, with rates ranging from 30-100 percent. As a result, manual inspection of violations in the source code adds a significant overhead to developers without clear benefit. From the viewpoint of fault prevention, there is an even more ironic aspect to enforcing ineffective rules. Any modification of the software has a non-zero probability of introducing a new fault, and if this probability exceeds the reduction achieved by fixing the violation, the net result is an increased probability of faults in the software.

In this talk, we discuss an approach to use the software history of a project to customize a coding standard. In this approach, we focus on the fault prevention aspect of a standard, and link violations to known bugs using the versioning system and issue tracking database. This information can in turn be used to compute true positive rates for every rule, an indication of the likelihood with which a violation of that rule correctly predicts the location of a bug. The observed accuracy allows us to select an effective subset of the standard, as well as rank a list of violations. In the rule selection, we provide a trade-off analysis method that makes the dependency between the potential number of bugs found and the number of violations to inspect explicit.

We apply our approach to three embedded software projects from NXP using QA-C and the Misra C 2004 standard, and find sets of 9 to 36 effective fault-prediction rules. These sets reduce the number of violations to inspect by 63 to 95 percent, while still covering 64 to 86 percent of the bug locations covered by all violations.