A Holistic Overview of Software Engineering Research Strategies
- 1 May 2015
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2015 IEEE/ACM 3rd International Workshop on Conducting Empirical Studies in Industry
Abstract
Empirical research studies are the principal mechanism through which the software engineering research community studies and learns from software engineering practice. The focus on empirical studies has increased significantly in the past decade, more or less coinciding with the emergence of evidence-based software engineering, an idea that was proposed in 2004. As a consequence, the software engineering community is familiar with a range of empirical methods. However, while several overviews exist of popular empirical research methods, such as case studies and experiments, we lack a 'holistic' view of a more complete spectrum of research methods. Furthermore, while researchers will readily accept that all methods have inherent limitations, methods such as case study are still frequently critiqued for the lack of control that a researcher can exert in such a study, their use of qualitative data, and the limited generalizability that can be achieved. Controlled experiments are seen by many as yielding stronger evidence than case studies, but these can also be criticized due to the limited realism of the context in which they are conducted. We identify a holistic set of research methods and indicate their strengths and weaknesses in relation to various research elements.Keywords
This publication has 41 references indexed in Scilit:
- Borrowing from the Crowd: A Study of Recombination in Software Design CompetitionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Two's company, three's a crowd: a case study of crowdsourcing software developmentPublished by Association for Computing Machinery (ACM) ,2014
- Method and developer characteristics for effective agile method tailoringACM Transactions on Software Engineering and Methodology, 2010
- Guidelines for conducting and reporting case study research in software engineeringEmpirical Software Engineering, 2008
- Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same SystemIEEE Transactions on Software Engineering, 2008
- Realizing quality improvement through test driven development: results and experiences of four industrial teamsEmpirical Software Engineering, 2008
- Evaluating Pair Programming with Respect to System Complexity and Programmer ExpertiseIEEE Transactions on Software Engineering, 2007
- An Ethnographic Study of XP PracticeEmpirical Software Engineering, 2004
- RESEARCH METHODOLOGY IN MANAGEMENT: CURRENT PRACTICES, TRENDS, AND IMPLICATIONS FOR FUTURE RESEARCH.The Academy of Management Journal, 2000
- A case study of open source software developmentPublished by Association for Computing Machinery (ACM) ,2000