Optimal test distributions for software failure cost estimation

Abstract

In this article, we generalize the input-domain based software reliability measures by Nelson and by Weiss and Weyuker, introducing expected failure costs under the operational distribution as a measure for software unreliability. This approach incorporates in the reliability concept a distinction between different degrees of failure severity. It is shown how to estimate the proposed quantity by means of random testing, using the Importance Sampling technique from Rare Event Simulation. A test input distribution that yields an unbiased estimator with minimum variance is determined. The practical application of the presented method is outlined, and a detailed numerical example is given.