Browsing by Author "Garousi, Vahid"

Now showing 1 - 13 of 13

Citation Count: 20
A BIBLIOMETRIC/GEOGRAPHIC ASSESSMENT OF 40 YEARS OF SOFTWARE ENGINEERING RESEARCH (1969-2009)
(World Scientific Publ Co Pte Ltd, 2013) Garousi, Vahid; Ruhe, Guenther; Software Engineering
Bibliometric rankings are quite common in the field of software engineering. For example, there are a series of ranking repeated every year which identify the top researchers and institutions at the international level in the field. There are also other studies to determine the most cited articles in software engineering journals, the most popular research topics in this area, or identify the top researchers and institutions in regional levels. However, there exists no existing bibliometric quantitative analysis of publications in the area of software engineering (SE), including relative and absolute growth in the number of all SE publications as well as an analysis among countries. This is the main goal and motivation of this article. Besides, this study intends to provide an overall quantitative trend of the software engineering papers, and compare that trend to research output in other areas of science. The bibliometric study reported in this paper is motivated by the fact that understanding the amount of geographical research contributions to the field of software engineering can help identify different countries' level of commitment to support research activities in this area over years. We analyze how the contribution levels of top-ranked countries have changed over the years and how SE compares to other disciplines of engineering and science. Among the most interesting findings of this study are: (1) Over 40 years, in total about 60% of the SE literature has been contributed by only 7% of all countries, (2) the SE research output of different countries does not necessarily correlate with their GDPs, (3) the share of contributions to the SE discipline by the American researchers has declined from 71.43% ( in 1980) to 14.90% ( in 2008), and ( 4) China is the country with the biggest share growth in the number of publications ( from 0.82% of the entire SE publications in 1991 to 13.82% in 2009).
Citation Count: 130
Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review
(Elsevier, 2016) Garousi, Vahid; Petersen, Kai; Ozkan, Baris; Information Systems Engineering; Software Engineering
Context: The global software industry and the software engineering (SE) academia are two large communities. However, unfortunately, the level of joint industry-academia collaborations in SE is still relatively very low, compared to the amount of activity in each of the two communities. It seems that the two 'camps' show only limited interest/motivation to collaborate with one other. Many researchers and practitioners have written about the challenges, success patterns (what to do, i.e., how to collaborate) and anti-patterns (what not do do) for industry-academia collaborations. Objective: To identify (a) the challenges to avoid risks to the collaboration by being aware of the challenges, (b) the best practices to provide an inventory of practices (patterns) allowing for an informed choice of practices to use when planning and conducting collaborative projects. Method: A systematic review has been conducted. Synthesis has been done using grounded-theory based coding procedures. Results: Through thematic analysis we identified 10 challenge themes and 17 best practice themes. A key outcome was the inventory of best practices, the most common ones recommended in different contexts were to hold regular workshops and seminars with industry, assure continuous learning from industry and academic sides, ensure management engagement, the need for a champion, basing research on real world problems, showing explicit benefits to the industry partner, be agile during the collaboration, and the co-location of the researcher on the industry side. Conclusion: Given the importance of industry-academia collaboration to conduct research of high practical relevance we provide a synthesis of challenges and best practices, which can be used by researchers and practitioners to make informed decisions on how to structure their collaborations. (C) 2016 Elsevier B.V. All rights reserved.
Citation Count: 69
Cost, benefits and quality of software development documentation: A systematic mapping
(Elsevier Science inc, 2015) Zhi, Junji; Garousi-Yusifoglu, Vahid; Sun, Bo; Garousi, Golara; Shahnewaz, Shawn; Ruhe, Guenther; Software Engineering
Context: Software documentation is an integral part of any software development process. Researchers and practitioners have expressed concerns about costs, benefits and quality of software documentation in practice. On the one hand, there is a lack of a comprehensive model to evaluate the quality of documentation. On the other hand, researchers and practitioners need to assess whether documentation cost outweighs its benefit. Objectives: In this study, we aim to summarize the existing literature and provide an overview of the field of software documentation cost, benefit and quality. Method: We use the systematic-mapping methodology to map the existing body of knowledge related to software documentation cost, benefit and quality. To achieve our objectives, 11 Research Questions (RQ) are raised. The primary papers are carefully selected. After applying the inclusion and exclusion criteria, our study pool included a set of 69 papers from 1971 to 2011. A systematic map is developed and refined iteratively. Results: We present the results of a systematic mapping covering different research aspects related to software documentation cost, benefit and quality (RQ1-11). Key findings include: (1) validation research papers are dominating (27 papers), followed by solution proposals (21 papers). (2) Most papers (61 out of 69) do not mention the development life-cycle model explicitly. Agile development is only mentioned in 6 papers. (3) Most papers include only one "System under Study" (SUS) which is mostly academic prototype. The average number of participants in survey-based papers is 106, the highest one having approximately 1000 participants. (4) In terms of focus of papers, 50 papers focused on documentation quality, followed by 37 papers on benefit, and 12 papers on documentation cost. (5) The quality attributes of documentation that appear in most papers are, in order: completeness, consistency and accessibility. Additionally, improved meta-models for documentation cost, benefit and quality are also presented. Furthermore, we have created an online paper repository of the primary papers analyzed and mapped during this study. Conclusion: Our study results show that this research area is emerging but far from mature. Firstly, documentation cost aspect seems to have been neglected in the existing literature and there are no systematic methods or models to measure cost. Also, despite a substantial number of solutions proposed during the last 40 years, more and stronger empirical evidences are still needed to enhance our understanding of this area. In particular, what we expect includes (1) more validation or evaluation studies; (2) studies involving large-scale development projects, or from large number of study participants of various organizations; (3) more industry-academia collaborations; (4) more estimation models or methods to assess documentation quality, benefit and, especially, cost. (C) 2014 Elsevier Inc. All rights reserved.
Citation Count: 8
Cross-factor analysis of software engineering practices versus practitioner demographics: An exploratory study in Turkey
(Elsevier Science inc, 2016) Garousi, Vahid; Coskuncay, Ahmet; Demirors, Onur; Yazici, Ali; Software Engineering
Context: Understanding the types of software engineering practices and techniques used in the industry is important. There is a wide spectrum in terms of the types and maturity of software engineering practices conducted in each software team and company. Furthermore, it is important to understand the cross-factor relationship of software engineering practices and practitioner demographics including their companies and projects, e.g., is it the case that practitioners developing software for sectors such as military would utilize software size measurement approaches more, or use different software development methodologies, compared to practitioners developing software for other sectors?, and what kinds of practices are conducted by practitioners in small versus large companies? Objective: Our objective is to get an understanding into the cross-factor correlation of various software engineering practices versus practitioner demographics including their companies and projects (e.g., target industry, size and work experience). Such an understanding will enable us to identify patterns and pinpoint special issues which should be studied and addressed in the context of each specific demographic (e.g., small versus large companies). Thus, we decided to conduct an exploratory study in this area and collected real industrial data in the context of Turkey which has a vibrant software industry. Method: To achieve the above objective, we use the data from a recent Turkish-wide survey of software engineering practices which was systematically designed with 46 questions based on our past experience in the Canadian context and using the Software Engineering Body of Knowledge (SWEBOK). 202 practicing software engineers participated in the survey. We raise a set of 12 research questions about the cross-factor correlation of software engineering practices and practitioner demographics, and address them using statistical analysis. Results: The exploratory study results reveal important and interesting findings about cross-factor relationship of software engineering practices and practitioner demographics. Among some of the most interesting findings are the followings: (1) By analyzing the trends, we were first surprised to see that as a practitioner gets more years of work experience, against what one would expect, s/he experiences more challenges in Software Development Life-Cycle (SDLC) phases and SE tasks; (2) Almost 55% of participants measure software size; (3) Agile/lean development is used the least (16%) by the participants working in the companies serving the military and defense sector; (4) Usage of waterfall is low among participants employed by small sized companies whereas Agile/lean development is relatively popular among this class of participants; and (5) As company size increases, usage of spiral development slightly increases, whereas usage of extreme programming practices decreases. Conclusion: The results of this exploratory study will be useful to software engineering professionals and researchers both in Turkey and world-wide by revealing the cross-factor relationship of software engineering practices versus practitioner demographics. The study raises several new research directions, e.g., (1) Why are not many practitioners using any size nor project estimation metrics and how these fundamental engineering approaches could be utilized more frequently?, (2) What are the best practices, success stories and challenging experiences in using SE tools?, and (3) Why is Agile lean development is not popular in the military and defense sector and how these approaches could be utilized more frequently. (C) 2015 Elsevier Inc. All rights reserved.
Citation Count: 14
A Search-based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study
(Ieee, 2014) Amannejad, Yasaman; Garousi, Vahid; Irving, Rob; Sahaf, Zahra; Software Engineering
Test automation is a widely-used approach to reduce the cost of manual software testing. However, if it is not planned or conducted properly, automated testing would not necessarily be more cost effective than manual testing. Deciding what parts of a given System Under Test (SUT) should be tested in an automated fashion and what parts should remain manual is a frequently-asked and challenging question for practitioner testers. In this study, we propose a search-based approach for deciding what parts of a given SUT should be tested automatically to gain the highest Return On Investment (ROI). This work is the first systematic approach for this problem, and significance of our approach is that it considers automation in the entire testing process (i.e., from test-case design, to test scripting, to test execution, and test result evaluation). The proposed approach has been applied in an industrial setting in the context of a software product used in the oil and gas industry in Canada. Among the results of the case study is that, when planned and conducted properly using our decision-support approach, test automation provides the highest ROI. In this study, we show that if automation decision is taken effectively, test-case design, test execution, and test evaluation can result in about 307%, 675%, and 41% ROI in 10 rounds of using automated test suites.
Citation Count: 84
Smells in software test code: A survey of knowledge in industry and academia
(Elsevier Science inc, 2018) Garousi, Vahid; Kucuk, Baris; Software Engineering
As a type of anti-pattern, test smells are defined as poorly designed tests and their presence may negatively affect the quality of test suites and production code. Test smells are the subject of active discussions among practitioners and researchers, and various guidelines to handle smells are constantly offered for smell prevention, smell detection, and smell correction. Since there is a vast grey literature as well as a large body of research studies in this domain, it is not practical for practitioners and researchers to locate and synthesize such a large literature. Motivated by the above need and to find out what we, as the community, know about smells in test code, we conducted a 'multivocal' literature mapping (classification) on both the scientific literature and also practitioners' grey literature. By surveying all the sources on test smells in both industry (120 sources) and academia (46 sources), 166 sources in total, our review presents the largest catalogue of test smells, along with the summary of guidelines/techniques and the tools to deal with those smells. This article aims to benefit the readers (both practitioners and researchers) by serving as an "index" to the vast body of knowledge in this important area, and by helping them develop high-quality test scripts, and minimize occurrences of test smells and their negative consequences in large test automation projects. (C) 2017 Elsevier Inc. All rights reserved.
Citation Count: 62
Software test maturity assessment and test process improvement: A multivocal literature review
(Elsevier, 2017) Garousi, Vahid; Felderer, Michael; Hacaloglu, Tuna; Information Systems Engineering; Software Engineering
Context: Software testing practices and processes in many companies are far from being mature and are usually conducted in ad-hoc fashions. Such immature practices lead to various negative outcomes, e.g., ineffectiveness of testing practices in detecting all the defects, and cost and schedule overruns of testing activities. To conduct test maturity assessment (TMA) and test process improvement (TPI) in a systematic manner, various TMA/TPI models and approaches have been proposed. Objective: It is important to identify the state-of-the-art and the-practice in this area to consolidate the list of all various test maturity models proposed by practitioners and researchers, the drivers of TMA/TPI, the associated challenges and the benefits and results of TMA/TPI. Our article aims to benefit the readers (both practitioners and researchers) by providing the most comprehensive survey of the area, to this date, in assessing and improving the maturity of test processes. Method: To achieve the above objective, we have performed a Multivocal Literature Review (MLR) study to find out what we know about TMA/TPI. A MLR is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). We searched the academic literature using the Google Scholar and the grey literature using the regular Google search engine. Results: Our MLR and its results are based on 181 sources, 51 (29%) of which were grey literature and 130 (71%) were formally published sources. By summarizing what we know about TMA/TPI, our review identified 58 different test maturity models and a large number of sources with varying degrees of empirical evidence on this topic. We also conducted qualitative analysis (coding) to synthesize the drivers, challenges and benefits of TMA/TPI from the primary sources. Conclusion: We show that current maturity models and techniques in TMA/TPI provides reasonable advice for industry and the research community. We suggest directions for follow-up work, e.g., using the findings of this MLR in industry-academia collaborative projects and empirical evaluation of models and techniques in the area of TMA/TPI as reported in this article. (C) 2017 Elsevier B.V. All rights reserved.
Citation Count: 37
Software test-code engineering: A systematic mapping
(Elsevier, 2015) Yusifoglu, Vahid Garousi; Amannejad, Yasaman; Can, Aysu Betin; Software Engineering
Context: As a result of automated software testing, large amounts of software test code (script) are usually developed by software teams. Automated test scripts provide many benefits, such as repeatable, predictable, and efficient test executions. However, just like any software development activity, development of test scripts is tedious and error prone. We refer, in this study, to all activities that should be conducted during the entire lifecycle of test-code as Software Test-Code Engineering (STCE). Objective: As the STCE research area has matured and the number of related studies has increased, it is important to systematically categorize the current state-of-the-art and to provide an overview of the trends in this field. Such summarized and categorized results provide many benefits to the broader community. For example, they are valuable resources for new researchers (e.g., PhD students) aiming to conduct additional secondary studies. Method: In this work, we systematically classify the body of knowledge related to STCE through a systematic mapping (SM) study. As part of this study, we pose a set of research questions, define selection and exclusion criteria, and systematically develop and refine a systematic map. Results: Our study pool includes a set of 60 studies published in the area of STCE between 1999 and 2012. Our mapping data is available through an online publicly-accessible repository. We derive the trends for various aspects of STCE. Among our results are the following: (1) There is an acceptable mix of papers with respect to different contribution facets in the field of STCE and the top two leading facets are tool (68%) and method (65%). The studies that presented new processes, however, had a low rate (3%), which denotes the need for more process-related studies in this area. (2) Results of investigation about research facet of studies and comparing our result to other SM studies shows that, similar to other fields in software engineering, STCE is moving towards more rigorous validation approaches. (3) A good mixture of STCE activities has been presented in the primary studies. Among them, the two leading activities are quality assessment and co-maintenance of test-code with production code. The highest growth rate for co-maintenance activities in recent years shows the importance and challenges involved in this activity. (4) There are two main categories of quality assessment activity: detection of test smells and oracle assertion adequacy. (5) JUnit is the leading test framework which has been used in about 50% of the studies. (6) There is a good mixture of SUT types used in the studies: academic experimental systems (or simple code examples), real open-source and commercial systems. (7) Among 41 tools that are proposed for STCE, less than half of the tools (45%) were available for download. It is good to have this percentile of tools to be available, although not perfect, since the availability of tools can lead to higher impact on research community and industry. Conclusion: We discuss the emerging trends in STCE, and discuss the implications for researchers and practitioners in this area. The results of our systematic mapping can help researchers to obtain an overview of existing STCE approaches and spot areas in the field that require more attention from the research community. (C) 2014 Elsevier B.V. All rights reserved.
Citation Count: 0
Üniversite Yazilim Mühendisligi programlari müfredatinin SWEBOK kilavuzu kullanilarak degerlendirme ve iyilestirilmesi: Türkiye'de bir vaka çalismasi
(CEUR-WS, 2015) Garousi,V.; Mishra,A.; Yazici,A.; Software Engineering
[No abstract available]
Citation Count: 40
Usage and usefulness of technical software documentation: An industrial case study
(Elsevier, 2015) Garousi, Golara; Garousi-Yusifoglu, Vahid; Ruhe, Guenther; Zhi, Junji; Moussavi, Mahmoud; Smith, Brian; Software Engineering
Context: Software documentation is an integral part of any software development process. However, software practitioners are often concerned about the value, degree of usage and usefulness of documentation during development and maintenance. Objective: Motivated by the needs of NovAtel Inc. (NovAtel), a world-leading company developing software systems in support of global navigation satellite systems, and based on the results of a former systematic mapping study, we aimed at better understanding of the usage and the usefulness of various technical documents during software development and maintenance. Method: We utilized the results of a former systematic mapping study and performed an industrial case study at NovAtel. From the joint definition of the analysis goals, the research method incorporates qualitative and quantitative analysis of 55 documents (design, test and process related) and 1630 of their revisions. In addition, we conducted a survey on the usage and usefulness of documents. A total of 25 staff members from the industrial partner, all having a medium to high level of experience, participated in the survey. Results: In the context of the case study, a number of findings were derived. They include that (1) technical documentation was consulted least frequently for maintenance purpose and most frequently as an information source for development, (2) source code was considered most frequently as the preferred information source during software maintenance, (3) there is no significant difference between the usage of various documentation types during both development and maintenance, and (4) initial hypotheses stating that up-to-date information, accuracy and preciseness have the highest impact on usefulness of technical documentation. Conclusions: It is concluded that the usage of documentation differs for various purposes and it depends on the type of the information needs as well as the tasks to be completed (e.g., development and maintenance). The results have been confirmed to be helpful for the company under study, and the firm is currently implementing some of the recommendations given. (C) 2014 Elsevier B.V. All rights reserved.
Citation Count: 56
Web application testing: A systematic literature review
(Elsevier Science inc, 2014) Dogan, Serdar; Betin-Can, Aysu; Garousi, Vahid; Software Engineering
Context: The web has had a significant impact on all aspects of our society. As our society relies more and more on the web, the dependability of web applications has become increasingly important. To make these applications more dependable, for the past decade researchers have proposed various techniques for testing web-based software applications. Our literature search for related studies retrieved 193 papers in the area of web application testing, which have appeared between 2000 and 2013. Objective: As this research area matures and the number of related papers increases, it is important to systematically identify, analyze, and classify the publications and provide an overview of the trends and empirical evidence in this specialized field. Methods: We systematically review the body of knowledge related to functional testing of web application through a systematic literature review (SLR) study. This SLR is a follow-up and complimentary study to a recent systematic mapping (SM) study that we conducted in this area. As part of this study, we pose three sets of research questions, define selection and exclusion criteria, and synthesize the empirical evidence in this area. Results: Our pool of studies includes a set of 95 papers (from the 193 retrieved papers) published in the area of web application testing between 2000 and 2013. The data extracted during our SLR study is available through a publicly-accessible online repository. Among our results are the followings: (1) the list of test tools in this area and their capabilities, (2) the types of test models and fault models proposed in this domain, (3) the way the empirical studies in this area have been designed and reported, and (4) the state of empirical evidence and industrial relevance. Conclusion: We discuss the emerging trends in web application testing, and discuss the implications for researchers and practitioners in this area. The results of our SLR can help researchers to obtain an overview of existing web application testing approaches, fault models, tools, metrics and empirical evidence, and subsequently identify areas in the field that require more attention from the research community. (C) 2014 Elsevier Inc. All rights reserved.
Citation Count: 15
What We Know about Software Test Maturity and Test Process Improvement
(Ieee Computer Soc, 2018) Garousi, Vahid; Felderer, Michael; Hacaloglu, Tuna; Information Systems Engineering; Software Engineering
[No Abstract Available]
Citation Count: 26
When to automate software testing? decision support based on system dynamics: An industrial case study
(Association for Computing Machinery, 2014) Sahaf,Z.; Garousi,V.; Pfahl,D.; Irving,R.; Amannejad,Y.; Software Engineering
Software test processes are complex and costly. To reduce testing effort without compromising effectiveness and product quality, automation of test activities has been adopted as a popular approach in software industry. However, since test automation usually requires substantial upfront investments, automation is not always more cost-effective than manual testing. To support decision-makers in finding the optimal degree of test automation in a given project, we propose in this paper a simulation model using the System Dynamics (SD) modeling technique. With the help of the simulation model, we can evaluate the performance of test processes with varying degrees of automation of test activities and help testers choose the most optimal cases. As the case study, we describe how we used our simulation model in the context of an Action Research (AR) study conducted in collaboration with a software company in Calgary, Canada. The goal of the study was to investigate how the simulation model can help decision-makers decide whether and to what degree the company should automate their test processes. As a first step, we compared the performances of the current fully manual testing with several cases of partly automated testing as anticipated for implementation in the partner company. The development of the simulation model as well as the analysis of simulation results helped the partner company to get a deeper understanding of the strengths and weaknesses of their current test process and supported decision-makers in the cost effective planning of improvements of selected test activities. © 2014 ACM.