Faults' context matters
- 3 September 2007
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM) in Fourth international workshop on Software quality assurance in conjunction with the 6th ESEC/FSE joint meeting - SOQUA '07
- p. 112-115
- https://doi.org/10.1145/1295074.1295096
Abstract
When choosing a testing technique, practitioners want to know which one will detect the faults that matter most to them in the programs that they plan to test. Do empirical evaluations of testing techniques provide this information? More often than not, they report how many faults in a carefully chosen "representative" sample the evaluated techniques detect. But the population of faults that such a sample would represent depends heavily on the faults' context or environment---as does the cost of failing to detect those faults. If empirical studies are to provide information that a practitioner can apply outside the context of the study, they must characterize the faults studied in a way that translates across contexts. A testing technique's fault-detecting abilities could then be interpreted relative to the fault characterization. In this paper, we present a list of criteria that a fault characterization must meet in order to be fit for this task, and we evaluate several well-known fault characterizations against the criteria. Two families of characterizations are found to satisfy the criteria: those based on graph models of programs and those based on faults' detection by testing techniques.Keywords
This publication has 8 references indexed in Scilit:
- Designing and comparing automated test oracles for GUI-based software applicationsACM Transactions on Software Engineering and Methodology, 2007
- Is mutation an appropriate tool for testing experiments?Published by Association for Computing Machinery (ACM) ,2005
- Can we measure software testing effectiveness?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- An approach to fault modeling and fault seeding using the program dependence graphJournal of Systems and Software, 1997
- A semantic model of program faultsPublished by Association for Computing Machinery (ACM) ,1996
- Operational profiles in software-reliability engineeringIEEE Software, 1993
- Comparing the Effectiveness of Software Testing StrategiesIEEE Transactions on Software Engineering, 1987
- Reliability of the Path Analysis Testing StrategyIEEE Transactions on Software Engineering, 1976