Time Complexity Comparison of Stopping at First Failure and Completely Running the Test

Abstract

For a given test set, we compare two test application techniques, "stopping at first failure" and "complete run", respectively. The former stops the test whenever a failure is encountered. The latter however, carries on till the end. Our comparison concludes that although the first technique is generally faster, there is no strong evidence to support a complete rejection of the second technique for it provides statistics useful in test planning. Results support an interchangeable application of both techniques with more frequent use of stopping at first failure. Even though there are numerous studies to decide how to prioritize and select test cases (TC), or to examine dependencies of TCs and codes, the presented comparison of the two test application techniques for a given "test set" has the potential to reduce the total run time of the test.