Reducing qualitative human oracle costs associated with automatically generated test data
- 13 July 2010
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
Due to the frequent non-existence of an automated oracle, test cases are often evaluated manually in practice. However, this fact is rarely taken into account by automatic test data generators, which seek to maximise a program's structural coverage only. The test data produced tends to be of a poor fit with the program's operational profile. As a result, each test case takes longer for a human to check, because the scenarios that arbitrary-looking data represent require time and effort to understand. This short paper proposes methods to extracting knowledge from programmers, source code and documentation and its incorporation into the automatic test data generation process so as to inject the realism required to produce test cases that are quick and easy for a human to comprehend and check. The aim is to reduce the so-called qualitative human oracle costs associated with automatic test data generation. The potential benefits of such an approach are demonstrated with a simple case study.Keywords
This publication has 12 references indexed in Scilit:
- Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost ProblemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid SearchIEEE Transactions on Software Engineering, 2009
- Increasing diversity: Natural language measures for software fault predictionJournal of Systems and Software, 2009
- Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Efficient unit test case minimizationPublished by Association for Computing Machinery (ACM) ,2007
- A multi-objective approach to search-based test data generationPublished by Association for Computing Machinery (ACM) ,2007
- CUTEPublished by Association for Computing Machinery (ACM) ,2005
- DARTACM SIGPLAN Notices, 2005
- Automated software test data generationIEEE Transactions on Software Engineering, 1990
- Symbolic execution and program testingCommunications of the ACM, 1976