Evolution of Software Testing Strategies and Trends: Semantic Content Analysis of Software Research Corpus of the Last 40 Years

Abstract

From the early days of computer systems to the present, software testing has been considered as a crucial process that directly affects the quality and reliability of software-oriented products and services. Accordingly, there is a huge amount of literature regarding the improvement of software testing approaches. However, there are limited reviews that show the whole picture of the software testing studies covering the topics and trends of the field. This study aims to provide a general figure reflecting topics and trends of software testing by analyzing the majority of software testing articles published in the last 40 years. A semi-automated methodology is developed for the analysis of software testing corpus created from core publication sources. The methodology of the study is based on the implementation of probabilistic topic modeling approach to discover hidden semantic patterns in the 14,684 published articles addressing software testing issues between 1980 and 2019. The results revealed 42 topics of the field, highlighting five software development ages, namely specification, detection, generation, evaluation, and prediction. The recent accelerations of the topics also showed a trend toward prediction-based software testing actions. Additionally, a higher trend on the topics concerning "Security Vulnerability", "Open Source" and "Mobile Application" was identified. This study showed that the current trend of software testing is towards prediction-based testing strategies. Therefore, the findings of this study may provide valuable insights for the industry and software communities to be prepared for the possible changes in the software testing procedures using prediction-based approaches.