5 results
Search Results
Now showing 1 - 5 of 5
Article Citation - WoS: 16What We Know About Software Test Maturity and Test Process Improvement(Ieee Computer Soc, 2018) Garousi, Vahid; Felderer, Michael; Hacaloglu, Tuna[No Abstract Available]Conference Object Citation - WoS: 16Citation - Scopus: 30A Search-Based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study(Ieee, 2014) Amannejad, Yasaman; Garousi, Vahid; Irving, Rob; Sahaf, ZahraTest automation is a widely-used approach to reduce the cost of manual software testing. However, if it is not planned or conducted properly, automated testing would not necessarily be more cost effective than manual testing. Deciding what parts of a given System Under Test (SUT) should be tested in an automated fashion and what parts should remain manual is a frequently-asked and challenging question for practitioner testers. In this study, we propose a search-based approach for deciding what parts of a given SUT should be tested automatically to gain the highest Return On Investment (ROI). This work is the first systematic approach for this problem, and significance of our approach is that it considers automation in the entire testing process (i.e., from test-case design, to test scripting, to test execution, and test result evaluation). The proposed approach has been applied in an industrial setting in the context of a software product used in the oil and gas industry in Canada. Among the results of the case study is that, when planned and conducted properly using our decision-support approach, test automation provides the highest ROI. In this study, we show that if automation decision is taken effectively, test-case design, test execution, and test evaluation can result in about 307%, 675%, and 41% ROI in 10 rounds of using automated test suites.Review Citation - WoS: 65Citation - Scopus: 80Software Test Maturity Assessment and Test Process Improvement: a Multivocal Literature Review(Elsevier, 2017) Garousi, Vahid; Felderer, Michael; Hacaloglu, TunaContext: Software testing practices and processes in many companies are far from being mature and are usually conducted in ad-hoc fashions. Such immature practices lead to various negative outcomes, e.g., ineffectiveness of testing practices in detecting all the defects, and cost and schedule overruns of testing activities. To conduct test maturity assessment (TMA) and test process improvement (TPI) in a systematic manner, various TMA/TPI models and approaches have been proposed. Objective: It is important to identify the state-of-the-art and the-practice in this area to consolidate the list of all various test maturity models proposed by practitioners and researchers, the drivers of TMA/TPI, the associated challenges and the benefits and results of TMA/TPI. Our article aims to benefit the readers (both practitioners and researchers) by providing the most comprehensive survey of the area, to this date, in assessing and improving the maturity of test processes. Method: To achieve the above objective, we have performed a Multivocal Literature Review (MLR) study to find out what we know about TMA/TPI. A MLR is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). We searched the academic literature using the Google Scholar and the grey literature using the regular Google search engine. Results: Our MLR and its results are based on 181 sources, 51 (29%) of which were grey literature and 130 (71%) were formally published sources. By summarizing what we know about TMA/TPI, our review identified 58 different test maturity models and a large number of sources with varying degrees of empirical evidence on this topic. We also conducted qualitative analysis (coding) to synthesize the drivers, challenges and benefits of TMA/TPI from the primary sources. Conclusion: We show that current maturity models and techniques in TMA/TPI provides reasonable advice for industry and the research community. We suggest directions for follow-up work, e.g., using the findings of this MLR in industry-academia collaborative projects and empirical evaluation of models and techniques in the area of TMA/TPI as reported in this article. (C) 2017 Elsevier B.V. All rights reserved.Article Citation - WoS: 114Citation - Scopus: 143Smells in Software Test Code: a Survey of Knowledge in Industry and Academia(Elsevier Science inc, 2018) Garousi, Vahid; Kucuk, BarisAs a type of anti-pattern, test smells are defined as poorly designed tests and their presence may negatively affect the quality of test suites and production code. Test smells are the subject of active discussions among practitioners and researchers, and various guidelines to handle smells are constantly offered for smell prevention, smell detection, and smell correction. Since there is a vast grey literature as well as a large body of research studies in this domain, it is not practical for practitioners and researchers to locate and synthesize such a large literature. Motivated by the above need and to find out what we, as the community, know about smells in test code, we conducted a 'multivocal' literature mapping (classification) on both the scientific literature and also practitioners' grey literature. By surveying all the sources on test smells in both industry (120 sources) and academia (46 sources), 166 sources in total, our review presents the largest catalogue of test smells, along with the summary of guidelines/techniques and the tools to deal with those smells. This article aims to benefit the readers (both practitioners and researchers) by serving as an "index" to the vast body of knowledge in this important area, and by helping them develop high-quality test scripts, and minimize occurrences of test smells and their negative consequences in large test automation projects. (C) 2017 Elsevier Inc. All rights reserved.Article Citation - WoS: 8Citation - Scopus: 15Cross-Factor Analysis of Software Engineering Practices Versus Practitioner Demographics: an Exploratory Study in Turkey(Elsevier Science inc, 2016) Garousi, Vahid; Coskuncay, Ahmet; Demirors, Onur; Yazici, AliContext: Understanding the types of software engineering practices and techniques used in the industry is important. There is a wide spectrum in terms of the types and maturity of software engineering practices conducted in each software team and company. Furthermore, it is important to understand the cross-factor relationship of software engineering practices and practitioner demographics including their companies and projects, e.g., is it the case that practitioners developing software for sectors such as military would utilize software size measurement approaches more, or use different software development methodologies, compared to practitioners developing software for other sectors?, and what kinds of practices are conducted by practitioners in small versus large companies? Objective: Our objective is to get an understanding into the cross-factor correlation of various software engineering practices versus practitioner demographics including their companies and projects (e.g., target industry, size and work experience). Such an understanding will enable us to identify patterns and pinpoint special issues which should be studied and addressed in the context of each specific demographic (e.g., small versus large companies). Thus, we decided to conduct an exploratory study in this area and collected real industrial data in the context of Turkey which has a vibrant software industry. Method: To achieve the above objective, we use the data from a recent Turkish-wide survey of software engineering practices which was systematically designed with 46 questions based on our past experience in the Canadian context and using the Software Engineering Body of Knowledge (SWEBOK). 202 practicing software engineers participated in the survey. We raise a set of 12 research questions about the cross-factor correlation of software engineering practices and practitioner demographics, and address them using statistical analysis. Results: The exploratory study results reveal important and interesting findings about cross-factor relationship of software engineering practices and practitioner demographics. Among some of the most interesting findings are the followings: (1) By analyzing the trends, we were first surprised to see that as a practitioner gets more years of work experience, against what one would expect, s/he experiences more challenges in Software Development Life-Cycle (SDLC) phases and SE tasks; (2) Almost 55% of participants measure software size; (3) Agile/lean development is used the least (16%) by the participants working in the companies serving the military and defense sector; (4) Usage of waterfall is low among participants employed by small sized companies whereas Agile/lean development is relatively popular among this class of participants; and (5) As company size increases, usage of spiral development slightly increases, whereas usage of extreme programming practices decreases. Conclusion: The results of this exploratory study will be useful to software engineering professionals and researchers both in Turkey and world-wide by revealing the cross-factor relationship of software engineering practices versus practitioner demographics. The study raises several new research directions, e.g., (1) Why are not many practitioners using any size nor project estimation metrics and how these fundamental engineering approaches could be utilized more frequently?, (2) What are the best practices, success stories and challenging experiences in using SE tools?, and (3) Why is Agile lean development is not popular in the military and defense sector and how these approaches could be utilized more frequently. (C) 2015 Elsevier Inc. All rights reserved.
