Generating small combinatorial test suites to cover input-output relationships

Abstract

In this paper, we consider a problem that arises in black box testing: generating small test suites (i.e., sets of test cases) where the combinations that have to be covered are specified by input-output parameter relationships of a software system. That is, we only consider combinations of input parameters that affect an output parameter. We also do not assume that the input parameters have the same number of values. To solve this problem, we revisit the greedy algorithm for test generation and show that the size of the test suite it generates is within a logarithmic factor of the optimal. Unfortunately, the algorithm's main weaknesses are its time and space requirements for construction. To address this issue, we present a problem reduction technique that makes the greedy algorithm or any other test suite generation method more efficient if the reduction in size is significant.