Statistical Usage Testing at Different Levels of Testing

Abstract

Statistical Usage Testing (SUT) is the testing technique defined in Cleanroom Software Engineering model [Runeson, 93]. Cleanroom Software Engineering model is a theory based and team oriented model that is based on development and certification of software in increments using statistical quality control [Linger 96]. SUT is a black box testing technique and concentrates on how the software completes its required function from the user's perspective [Runeson, 93]. SUT is carried out by developing usage models and assigning usage probabilities. Testing is carried out on usage models by performing statistical tests which are random sequences [Trammel 95]. Statistical testing can be viewed as a statistical experiment where random test cases are selected from all the usage models [Trammel 95]. This paper demonstrates the process and benefits of applying SUT at different levels of testing. Levels of testing include Unit level, Integration level, System level and Acceptance level. SUT is generally performed at System level and Unit testing is not the part of SUT. Unit testing makes it easier to access code and debug human errors. Detecting errors at an early stage helps reducing cost and effort. The paper proposes to allow Unit testing in Cleanroom Software Engineering Model, thus making it more flexible and suitable for varied applications. Unit testing is essentially performed to ensure that the code is working correctly and meets the user specifications [istqb, 15]. Errors may also exist when modules are integrated because of interchange of data and control information between various modules. Integration testing is performed when the modules are combined together to check their behaviour and functionality after integration. Once the Integration testing phase gets successfully completed, System testing is performed on the whole system [test-institute, 15]. The paper makes use of Student record software to demonstrate the process of performing SUT at different levels. In addition to performing SUT at System level, this paper helps in understanding the advantages of applying SUT at Unit level and Integration level.