International Business Machines Corporation
System testing infrastructure with hidden variable, hidden attribute, and hidden value detection

Abstract:

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (S.sub.AV) is computed based on the binary execution results. The method further includes outputting, to a user, in response to the success rate S.sub.AV of the attribute-value pair being below a predetermined threshold, an identification of one or more testcases that use the attribute-value pair, wherein the one or more testcases are to be used for diagnosing a soft failure associated with the SUT.