next up previous contents index
Next: Now what is a Up: Testing Software Previous: Advantages and Disadvantages of

When to stop Testing

  One of the most difficult questions out of the field of software testing is: ``When is a good point to bring the test to an end, and are you ever throughgif?'' There are several reasons why it is difficult to find a good answer. Once more psychology  plays an important role, especially if the ending condition is defined before the tests are started. Furthermore it can be assumed that -- if the program is not very tiny -- there is always one more bug. In other words, it is basically impossible to write a bigger, faultless program. A rule of thumb that can be used is that you should not stop testing as long as you keep finding bugs  fairly regularlygif. On the other hand, if you never start selling your product, you will sooner or later run out of business. So despite the danger of an image loss due to annoying faults of your program which you did not find, you have to start marketing your product.

These considerations lead to the most likely common and nevertheless absolutely senseless ending condition: testing ends after a certain period of time has elapsed. In other words, the tests end on a fixed date. The uselessness of this condition is buried in the human's purposeful behavior. A tester will normally not do his best if he gets a goal  like ``test n days'' or ``test until you cannot find any bugs  anymore''. Those conditions share a similar weakness. For instance, given the last one, the tester will unintentionally run tests which are less likely to detect a bug, instead of doing good, intelligent, and destructive  tests.

A better goal would be ``find n faults'', but even though it is far away from perfect, it contains the following risk: How do you find a good approximation for n? If it is too low, several mistakes will not be found; and if it is too high, psychological problems have to be faced again. If the tester gets the feeling that he is unable to fulfill the given goal, he will get frustrated, and the quality  of his tests will drop significantly.

Most likely, the best way out of this dilemma is to commit an external institution with the tests' implementation. This step has several advantages like a healthy and productive rivalry, increased motivation on both sides, a reduced influence of the developers on the testers, normally a higher qualification of the testers, and the constraint to write a better documentation about everything donegif.


next up previous contents index
Next: Now what is a Up: Testing Software Previous: Advantages and Disadvantages of

Ingo Melzer
Mon Aug 5 15:12:01 MET DST 1996