Defect leakage: a metric for quality code
It’s all very well to have a feel for how good your code is, but knowing and proving that you’re producing quality code is another matter. Catherine Powell tracks code quality, we don’t but should, and probably will be soon.
On the Abakas blog (a good read if you’re concerned about or responsible for software quality) she writes about what is perhaps the most useful metric that software developers can track: defect leakage. It’s the measure of the number of bugs found first in the field by the client (or users). There are a number of ways of reporting defect leakage: raw numbers is one, but one that is probably more useful is the ratio of bugs found by the client to bugs found by us. That gives an indication of how thorough our testing processes are.
A side effect is that to report this metric we need to track bugs in some detail, and this gives us the ability to break the metric down and pull more information out of the reports. Which areas have good coverage (as shown by a low leakage ratio)? Which areas have a high defect leakage ratio and need more attention before release?
Catherine Powell writes:
… measuring defect leakage requires you to really consider every customer incident, and that kind of introspection helps me see patterns in problems. It really points out where our coverage is lacking and, conversely, where our product is pretty good.
I like that!