International Business Machines Corporation
System testing infrastructure with hidden variable, hidden attribute, and hidden value detection

Abstract:

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using a set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, updating, for each execution of the set of testcases, for each testcase, a non-binary success rate (S.sub.T) based on the binary execution results. In response to a first success rate corresponding to a particular testcase being below a predetermined threshold, a second set of testcases is generated based on the test vectors. For each testcase, a second success rate (S.sub.T') is computed based on a second set of execution results of a second set of testcases. In response to the second success rate corresponding to the particular testcase being substantially same as the first success rate, a user is notified of a defect in modeling the SUT inputs.